Why NFL teams are making smarter fourth-down decisions

Lamar Jackson completed a quick pass across the middle to Sammy Watkins, who was tackled 1 yard short of a first down. That left the Baltimore Ravens facing a fourth-and-1 from their own 43-yard line. They led the Kansas City Chiefs, who had just used their last timeout, by one point. There was still 1:05 to play.

"Oh, boy," analyst Cris Collinsworth said during NBC's Sunday Night Football broadcast.

"You don't dare," play-by-play announcer Al Michaels said. "What do you do if you're Baltimore right now?"

"Don't discount the possibility," Collinsworth replied, "that they go for this."

"No, you don't," Michaels answered. "This is going to be one hell of a call, though."

As Michaels and Collinsworth conversed, Ravens head coach John Harbaugh could be seen shouting at Jackson, his quarterback, asking if he wanted to go for it. Yet even as Jackson returned to the field for the huddle, Michaels and Collinsworth weren't convinced Baltimore would actually run a scrimmage play.

The Ravens, of course, went for it. They put eight players along the offensive line. Jackson took a quick snap out of the shotgun and crept through a crease to pick up 2 yards. Game over.

"Go on with it, John Harbaugh," Collinsworth said. "The guts of a sailor."

Harbaugh's decision only seems gutsy compared to what has long passed for conventional football thinking. But the Ravens' choice to go for it was quite simply the right one. And that kind of fourth-down aggressiveness - rooted in hard data about the probabilities for success - is increasingly becoming the norm in the NFL, a league that has long been risk-averse and resistant to change.

Fourth-down aggressiveness is a staple of data-driven analysis; statistical models that weigh the changes in win probability based on the decision a team might make have existed for a long time. Yet only in recent years has the notion begun to catch on - largely because every NFL team now has an analytics department that works in conjunction with the football people, using that kind of statistical reasoning to influence decision-making.

The Ravens' options were to go for it or to punt. Both carried risks: Going for it and failing would mean the Chiefs would get the ball close to field-goal range. Punting meant giving the ever-dangerous Patrick Mahomes the ball back with roughly a minute to play. But going for it and succeeding virtually assured a Ravens victory.

Ben Baldwin's model, which crunches the numbers in real time, showed how going for it - by also measuring the likelihood of success - dramatically increased the Ravens' win probability:

The Ravens, in other words, had a 72% chance of making the first down, which gave them an estimated win probability of 84%. Failing to convert would drop their win probability to 42%, but converting would increase it to 100%. Punting still would have left them with a 66% win probability. But that's 18 percentage points less than deciding to go for it, given the high probability of successfully getting a first down in that spot.

Analytics firm EdjSports's model produced a similar result:

Fourth-down calculators - allowing anyone to assess the win probability of a given choice based on score differential, timeouts remaining, field position, down, and distance - have existed for many years. All of them incorporate the concept of expected points by using historical play-by-play data. (For more on expected points, read this.)

But Baldwin, who is committed to open-sourcing as much of this sort of thing as possible, took the concept of his calculator even further by sharing his methodology and by making the source code for his model public. Last year, he broke this all down in greater detail for The Athletic in an explainer that also noted the limitations of his model (it doesn't adjust for who a team's kicker or punter is, the weather, the possibility of a blocked field goal or punt, a punt return for a touchdown, or a turnover for a touchdown, among other factors).

Data-driven fourth-down models only produce recommendations, after all. It's ultimately up to coaches to consider that information in the context of a given situation.

The first study of fourth-down decision-making was a 1978 academic paper authored by Virgil Carter and Robert Machol, who created the concept of expected points a few years earlier. Their study, called "Optimal Strategies on Fourth Down," explored the impact on expected points of going for it versus kicking on fourth-and-short on the opponent's side of the field.

A hunch about fourth-down decision-making inspired Carter's and Machol's paper.

"Professor Machol always had the theory that (teams) weren't making good decisions when they were going in to score," Carter told theScore. "Fourth-and-1 at the 5-yard line going in, they'd kick a field goal. And he kept saying they should go for it because the probability of converting a fourth-and-1 is high enough and the value of having the ball there is worth enough that it outweighs turning it over, because even if you turn it over inside your opponent's 10-yard line, it's still plus value."

For a long time, the orthodox approach tended to involve coaches trusting their guts to play for field position, or deciding to trust their defenses. But as teams began to incorporate data scientists into their organizations, that thinking started to change.

From 2000-17, teams went for it on fourth-and-1 or fourth-and-2 just 32% of the time, according to the Wall Street Journal's Andrew Beaton and Ben Cohen. In 2018, that rate jumped to 45%. Football Outsiders' aggressiveness index reveals that the rate at which teams are going for it on fourth down over expectation has more than doubled since 2014, and overall attempts to go for it were 60% above expectation in 2020.

Even head coach Vic Fangio, whose Denver Broncos were last in FO's aggressiveness index in 2020, went for it on fourth down three times in a Week 1 win at the New York Giants. One of those was a fourth-and-7 from the Giants' 37 late in the first quarter - a scenario Baldwin's model described as a toss-up.

Using Baldwin's NFLfastR and NFL4th code and data repository, theScore found that teams went for it 16.8% of the time in 2019 when the model defined the decision as a toss-up, and that figure increased to 18.1% last year. Through Week 2 in 2021 (a small sample size, granted), coaches are going for it in 26.5% of those toss-up situations.

Fourth-down aggressiveness is not without risk, of course - and coaches know this.

There's the famous example from 2009 when Bill Belichick's New England Patriots went for it on fourth-and-2 from their own 28 with a little more than two minutes remaining while leading Indianapolis by six. The Pats failed to convert, and the Colts wound up winning by a point. The decision was widely mocked, but that night, Brian Burke wrote a short piece that used math to defend it for his blog that was picked up by the New York Times.

“That play definitely precipitated a lot of attention on analytics that never existed before," Burke later told The Ringer's Kevin Clark.

The Ravens have been at the forefront of the league's analytics movement. In the 2019-20 playoffs, while their eventual loss to the Tennessee Titans was still close, they went for it twice on fourth down and failed.

"Everybody remembers the ones that worked," the Broncos' Fangio told reporters last week. "It's nice when it works and everybody loves it, but there is another side of that story sometimes."

Baltimore had a nearly 30% of chance of failing to convert Sunday against the Chiefs. Had that happened, the conversation around Harbaugh's decision would likely have been much different. But the choice would have been no less correct.

Dom Cosentino is a senior features writer at theScore.