In 1954 Grim became the first Yankee rookie since Russ Ford in 1910 to win 20 games. He won a league-leading eight in relief to compile the only 20-win season ever of fewer than 200 innings (199). Coupled with his blazing fastball was a biting slider which damaged his arm and reduced his ability in subsequent years. In the 1955 WS he saved Game One, but in Game Five he gave up three Dodger homers to lose his only WS start.

Grim's arm troubles forced him solely into relief by 1957, when he led AL relievers with 12 wins, eight losses, and 19 saves. In Game Four of the '57 WS, he relieved Tommy Byrne in the bottom of the tenth after Byrne had hit Milwaukee's Nippy Jones with a pitch. Grim surrendered a double that tied the game, and then gave up a home run to Eddie Mathews. Grim was generally mediocre after leaving New York in 1958.