L.B. Jeffries (James Stewart) recuperates from a broken leg during a sweltering New York summer. As a successful photographer, he's known for taking difficult pictures no one else can get, including the one of an out-of-control race car which smashed his camera and broke his leg an instant after it was snapped. Jeffries lives in a small apartment, and spends his time looking out the rear window into the courtyard of the building; he can also see into the lives of all his neighbors, catching glimpses of their daily routines. It's the sort of thing only an invalid might do, watching them eat, clean, sleep and argue. There's the girl who exercises in her underwear, the married couple who sleep on their small balcony to beat the heat, the struggling songwriter working at his piano; and there's the salesman who lives across the courtyard from Jeffries, the one with the nagging bedridden wife. They seem to fight all too often.

Every day a therapist comes to visit Jeffries, dispensing her mature wisdom and berating him for sitting there all day spying on his neighbors. Stella (Thelma Ritter) tells him she can smell trouble coming. He should get his mind off his neighbors and think about marrying that beautiful girlfriend of his. Jeffries replies that he's not ready for marriage. Sure, she's a wonderful girl, but she's also a rich, successful socialite, and Jeffries lives the life of a war correspondent, always on the go, usually living out his suitcase and often in an unpleasant environment. It's not the life he wants to offer her. "Well" says Stella, "that girl is packed with love for you right down to her fingertips."

"That Girl" arrives shortly after Stella leaves. Lisa Carol Fremont (Grace Kelly) breezes in wearing a stunning satin dress, looking every inch the beautiful socialite she is, and obviously very much in love with Jeffries. They have dinner, but soon enough the conversation turns to the future, and they quarrel. Jeffries sees no way they can reconcile their different lifestyles, and she walks to the door, telling him goodbye. "When will I see you again?" asks Jeffries.

"Not for a long time," she replies sadly. "At least, not until tomorrow night."

The night drags by, and it's too hot for Jeffries to sleep. It starts to rain. He dozes in his wheelchair by the window, but notices activity across the yard. The salesman goes out carrying his heavy silver sample case, and Jeffries looks at his watch: it's 2:00am. The blinds in the bedroom are drawn, so Jeffries can't see the wife. Later, the salesman returns, lifting the case easily, as if it were empty. Twice more he goes out in the rain in the middle of the night, lugging the heavy case, but coming home with it lighter. Intrigued, Jeffries wonders what the salesman is doing, but he finally dozes off around daybreak.

Discussing the incident with Stella, and then later with Lisa, they all begin to watch the salesman. With the blinds now open, they can see that the wife is gone.