Efficiency in College Football

As the college football season advances into what promises to be an eventful month of November, a tireless debate rages on: Which teams are the best? For better or worse, the much-maligned Bowl Championship Series (BCS) rating system attempts to answer that question definitively. Every voter’s ballot may disagree and every computer system may weight certain factors over others, but when combined, doesn’t the BCS actually represent a kind of utopian model for finding consensus among a collection of disparate voices?

BCS-as-utopia may be overstating it, and it certainly doesn’t sit well with fans of the 117 teams not ranked #1 and #2 at season’s end. But at the risk of provoking the growing mob determined to bury the BCS where it may never be found again, can college football be served by yet another statistical rating system joining the conversation? A Division-1A playoff may be just around the corner or fifty years away. While we wait, more than 650 games are played between 119 teams every fall. When considering the question “Which teams are the best?,” can we better understand and evaluate the games that are settled on the field?

FEI Input Data - Game Efficiency

The criticism of the BCS computer elements is inseparably wed not just to a distrust of cold data analysis but to the severe handicaps imposed upon the computers themselves. Margin of Victory (MOV) data was eliminated from the BCS computers after the 2001 season in order to negate the impact of blowout wins and losses. However, even when MOV is used soundly by other ranking systems, can it be trusted? Is there not a difference between a 44-41 shootout and a 10-7 defensive battle? When win/loss outcomes or an unreliable stat are the only data input used by a computer ranking system, is it no wonder that the average fan distrusts the output? The Fremeau Efficiency Index (FEI) addresses this problem by first collecting better game data.

The game of football is basically divided into individual series of play, offense versus defense. A team on offense advances the ball until the series results in either a defensive stop (turnover, turnover on downs, punt, failed field goal, blocked kick, safety) or offensive score (field goal, touchdown), after which its opponent begins its own offensive series. This basic, alternating series structure is familiar to even the most novice fan, however note the method in which a typical game box score is published:

	1st	2nd	3rd	4th		F
USC	7	3	14	14		38
TEX	0	16	7	18		41

	USC	TEX
1st Downs	30	30
3rd down efficiency	8-14	3-11
4th down efficiency	1-3	1-2
Total Yards	574	556
Passing	365	267
Comp-Att	29-41	30-40
Yards per pass	8.3	6.7
Rushing	209	289
Rushing Attempts	41	36
Yards per rush	5.1	8.0
Penalties	5-30	4-34
Turnovers	2	1
Fumbles lost	1	1
Interceptions thrown	1	0
Possession	32:00	28:00

Take a moment to consider the value of this information. The scoring summary divides points scored by quarter. The team statistics divide yardage gained by passing and rushing. Possessions for each team are divided by total time elapsed while in control of the ball. Third and fourth down efficiency are given absent of drive context. Is it not strange that the basic division of play, the succession of possession series alternately played by the two teams, is totally ignored?

How would a fan having watched last January’s BCS championship game describe it to someone afterwards? By margin of victory? By a breakdown of team yardage? Wouldn’t the description more likely include important details like USC scoring touchdowns on each of its first four possessions of the second half, Vince Young’s heroics leading Texas’ final two possessions, and the game-hinging turnover on downs that set up the game-winning score?

Drive and play-by-play summaries are sometimes included as supporting information to the game box score, but these are presented in a comprehensive format that is difficult to synthesize. How well did a team maximize its own possessions and negate its opponent's possessions? It is the essential question in football, and it is answered statistically by Game Efficiency.

Game Efficiency quantifies the success rate of a team scoring while in possession of the ball and preventing scores while not in possession of the ball over the competitive course of a game. Since the success of a drive is contingent on the number of points it produces, there is a relationship between Game Efficiency and Margin of Victory, with two critical distinctions:

1. Game Efficiency represents not just an observed final outcome but how well each team played a given game to arrive at that outcome. In a sense, it is an enhancement of MOV, able to describe the difference between high-scoring shootouts and low-scoring defensive struggles, but also between a 17-14 ball-control game of only 15 possessions and a 17-14 triple-overtime game of 35 possessions.

2. Game Efficiency measures only the competitive possessions of a game, ignoring "garbage-time" scores and stops by both opponents. The only garbage-time adjustment options available for systems based on MOV are limited to arbitrarily assigned scoring ceilings or a formula of diminishing returns. Neither of these options can distinguish between, for example, a 24-point lead earned in the waning moments of the 4th quarter from a 24-point lead earned before halftime. By charting games series-by-series, Game Efficiency is able to make such distinctions, measure late-game scoring opportunities against scoring leads/deficits, and weight the conclusive possessions accordingly for the fairest measure of how well two teams played a given game.

Game Efficiency =
((Points For – Points Against)/7) / (Total Competitive Possessions/2)

	USC	TEX
Points	38	41
Competitive Possessions	13	12
Game Efficiency	-0.0343	0.0343

Processing the Data - FEI

Collecting Game Efficiency data from all games played thus far in the 2006 Division-1A college football season is a relatively basic exercise. But what do we do with the data once it is collected? How do we answer the question: Which teams are the best?

We could simply rank each team's average Game Efficiency over the course of the season. This method of processing the data, of course, does not take into account the quality of the opposition faced. A team could play extremely efficiently against a weak slate of opponents and hardly be considered “better” than a team that played less efficiently against a strong slate.

We could then adjust each Game Efficiency data point to account for the quality of opponent, and rank each team’s average Adjusted Game Efficiency over the course of the season. As valuable as this output (and subsequent-order versions of it) may be, it raises new questions that are more complex and completely unique to the challenge of evaluating 119 Division-1A teams: Can an efficiency margin recorded against the worst team in college football be effectively compared to an efficiency margin recorded against the best team? Are all data points and the results of all games played equally valuable?

The Fremeau Efficiency Index (FEI) weights the value of Adjusted Game Efficiency data by first evaluating the following criteria:

1. Who did you beat and how did you win those games?

2. Who did you lose to and how did you lose those games?

As the quality of the opponent decreases, the value of the first question receives less weight than the second.

FEI rewards teams for playing well against good teams, win or lose, and treats losing to poor teams more harshly than it rewards victories over poor teams.

Opening the debate

Are Game Efficiency and FEI the best way to determine the best teams in college football? The FEI Forecast will continue to predict winners of all Division-1A games each week based on the previous week’s rankings. But like any rating system in its infancy, several year’s worth of Game Efficiency data needs to be collected and evaluated in order to develop and advance the statistics and system going forward. Are we anywhere near utopia? If your vision of utopia allows for better and more in-depth statistical analysis and a healthy level of debate, we’re already there.