Ryan led the Falcons 80 yards for the 33-yard score after the Ravens had gone ahead with 1:05 left, on Joe Flacco's touchdown throw to Todd Heap.
Ryan finished 32-of-50 for 316 yards and three scores for Atlanta (7-2), which has won three in a row and holds the best record in the NFL heading into this weekend's action. The Falcons also lead idle New Orleans by one game for the NFC South lead.
"It's tough as a player to think big-picture," Ryan said. "You really have to focus on the small things, and it was a good win for us tonight. That's a good football team we played against. Nine games into it, we're in good position."
White, who was questionable with a knee ailment, caught 12 passes for 138 yards and two touchdowns.
Flacco completed 22-of-34 passes for 215 yards and three touchdowns, with one interception, as the Ravens (6-3) had a two-game win streak broken and fell out of the lead in the AFC North. They trail 6-2 Pittsburgh, which hosts New England on Sunday.
Atlanta led for most of the game, and about 3 1/2 minutes into the fourth quarter took a 20-7 advantage on Ryan's four-yard pass to White.
But the Ravens rallied, scoring on their next touch. Flacco converted a 3rd- and-8 in Atlanta territory with a 13-yard pass to Ray Rice, and connected with Derrick Mason for a six-yard touchdown several plays later.
The score had the Ravens within a possession with 5:42 left, and Baltimore's defense held, forcing a punt.
The Ravens took over with 2:57 left at their own 28-yard line, but got to midfield on the second play of the drive, thanks to Flacco's 22-yard toss to Mason.
Then, with under two minutes to play, Flacco found Rice for a 24-yard gain to get the ball to the Atlanta 11. After a two-yard run by Rice, Flacco gunned a pass over the middle for Heap, who made a leaning grab. The catch was upheld after a review, giving Baltimore a 21-20 edge with just 65 seconds to play.
The Ravens looked as though they would hold on, forcing Atlanta into a 3rd- and-10 on its first series. But Ryan connected with Michael Jenkins along the left sideline for a 24-yard gain.
Jenkins made the catch with his fingertips and continued to hold onto the ball that way as he dragged his right toe inbounds. The call was reviewed and upheld, giving the Falcons a first down at their own 44.
Ryan then went to White for a 15-yard gain, and spiked the ball to halt the clock with 34 seconds left. Atlanta still had one timeout, and Ryan went back to White on a quick slant, but the receiver couldn't hang on.
Ryan's third-down pass to tight end Tony Gonzalez also fell incomplete, but Ravens linebacker Tavares Gooden was called for pass interference on the play.
That gave Atlanta a fresh set of downs at the Baltimore 33. Ryan then found White open along the left side, made the connection, and White ran unimpeded into the end zone. White appeared to push Josh Wilson to free himself, and Wilson vigorously protested, but no flag was thrown.
The Falcons' two-point try failed, but the defense held steady to secure the comeback win.
"We took the lead there with a minute left, and with our defense on the field we were feeling pretty confident," Flacco said. "But like I said, they did a great job."
After a scoreless first quarter, Atlanta took the lead less than four minutes into the second. Ryan ended a 14-play, 91-yard drive with a 28-yard pass to running back Jason Snelling.
Later in the quarter, a punt-return fumble by Lardarius Webb gave the Falcons the ball at the Raven 43. Atlanta needed 12 plays to reach the 10, then settled for a 28-yard field goal by Matt Bryant with 17 seconds remaining.
The Falcons led by 10 at the break, and took advantage of another turnover shortly into the third. Brent Grimes picked off Flacco to set Atlanta up at the Baltimore 39. The Falcons weren't able to do much, and Bryant booted a 51- yarder for a 13-0 margin.
Flacco got the Ravens on the board on the ensuing drive, which he capped with a five-yard pass to Anquan Boldin.
Flacco's interception was his first since Week 4 against Pittsburgh...Atlanta converted 12-of-20 third downs...Rice had a game-high 59 rushing yards...Boldin led Ravens receivers with five catches for 50 yards...John Abraham had both of Atlanta's sacks, while Terrell Suggs had both of Baltimore's.