To those unfamiliar with Fringe, this may seem like an unusual scene. But it's pretty much just Walter being Walter.
Added:
On the episode "Grey Matters," the investigation leads Peter and the team inside a mental hospital.
Added:
Peter and Olivia visit a mental asylum on the episode "Grey Matters." That's where the investigation takes them.
Added:

Fringe Season 2 Episode 10 Quotes

Are you really trying to tell us that he is one of the heads they stole? Fine. Let's say for a second you're right. Frozen heads don't just get up and walk into places. I mean, the dead don't rise out of their graves and perform midnight brain surgery.

Peter

Security Guard: Are you Dr. Bishop?
Walter: Yes. And I'm perfectly sane.