Seven things the 2013 summer transfer window revealed about the State of Analytics

The 2013 European Summer Transfer Window has closed. The game of musical chairs is over. Fans and journalists have argued over the relative merits and drawbacks of their club’s signings, often referring to various statistics to make their point, from chances created to goals scored.

While proprietary concerns prevent us from looking with detail at how and in which ways analytics may have aided various teams in their search for the best players they can afford, there are still some lessons for us in the wake of the annual summer-long spree.

1. We have no idea what’s going on

By ‘we’ I mean the vast majority of us fans and journalists who are outside the confines of the club front offices. While we know a lot about the process of player transfers–protracted negotiations, helicopters landing in fields, agents fees, phantom third party bids meant to drive up the asking price–we don’t really know what clubs are looking for in the market, or whether scouts are armed with reliable predictive metrics that amateur analysts can only dream of, and if they are, whether they even have any real power in deciding who the club will sign.

This knowledge gap means we should be very careful when praising or criticizing clubs for various signings. We should also avoid making the lazy assumption that if a club buys players that aren’t already household names in England (like City did, for example), it means they based their decisions on in-house analytics. We probably shouldn’t even assume that failed bids are evidence of incompetence; despite the sums involved, it’s possible some clubs are more committed to strengthening the squad than others. Those who are not might prefer to chance less aggressive bids even at the risk of failure.

Finally, we shouldn’t assume that some or even most clubs are using analytics at all. Predictive individual player metrics may be a lost cause in football analytics, even when used in conjunction with traditional scouting methods; it might be far better to just chance it and rely on team analytics to aid the first team coach in doing their job.

2. The analytics-based transfers wouldn’t, in theory, generate headlines

Surprise! Clubs spend a lot of money on already good players because they tend to stay good, despite some very well-known exceptions (like Fernando Torres and Andy Carrol, as if you needed me to tell you that). That’s why the 34 league goal-scoring Gareth Bale went to Real Madrid for a record fee, and why Arsenal fans are pretty happy today that one of the best play-makers in world football, Mesut Ozil, is a Gunner. What clubs see might be what clubs get, without the need for elaborate metrics.

Presumably if a club has predictive player data that’s worth keeping secret, they wouldn’t be using it to make big name signings but instead to bet on younger unknown prospects at a reasonable price who they believe will develop into major assets down the road. Transfermarkt is a great resource for this…might be worth keeping an eye on the lesser lights to see how they fare in the long term, and to see if certain clubs have a better track record in graduating these players to the first team than others…

3. Transfer analytics should go beyond players

The big theme of this year’s market has been the perceived importance of the role of director of football to help guide the transfer process. Some journalists for example asserted that Man United and Arsenal’s several failed bids came down to the lack of a strong personality experienced in difficult transfer negotiations, while praising the work of Tottenham’s technical director Franco Baldini and Txiki Begiristain at Manchester City in getting things done.

It’s not really good policy however to hire a director of football based on anecdotal evidence from a single summer, and we have no idea how both Spurs and City’s acquisitions will fare over the coming season. But this summer revealed an opportunity to use data not just to evaluate players, but also to establish best practices in transfer dealings in general. As in: when is the most opportune time in the window to lodge bids? Is it better to low-ball in the hopes of a smaller fee, or ward off competition by offering above asking price? Are there clubs with a historically good record of buying good players on the cheap, only to sell them for a profit later on? If so, is there approach reproducible elsewhere? And, finally, is it better to delegate transfer negotiations to someone other than the manager or the chief executive?

Counter Attack has already looked at the possibility of front office analytics in the past…perhaps the next step would be for analysts to look at the transfer market as a separate entity altogether with its own attendant problems.

4. Sometimes not making a dumb transfer is more valuable than making a smart transfer

If you haven’t already, I recommend reading Phil Birnbaum’s post on Slate from July. He argues for the importance of analytics in not just making smart decisions, but avoiding dumb ones:

If the 1980 Expos had had a sabermetrics department, they could have spent hours trying to squeeze out a couple of extra runs by lineup management … but they would have been much, much better off figuring out that Rodney Scott’s offense was so bad, he shouldn’t have been a starter.

It works that way in your personal life, too. You can spend a lot of time and money picking out the perfect floral bouquet for your date, but you’re probably better off checking if you have bad breath and taking the porn out of the glove compartment.

If it’s true that sabermetrics helps teams win, I’d bet that most of the benefit comes from the “negative” side: having a framework that flags bad decisions before they get made.

Transfer deals often come down to luck of the draw, but in the end it might not matter much. Elite talent is spread thinly at the top, and most of the differences in valuation seem almost arbitrary. And all the probability tables in the world can’t prevent a broken leg, or a sudden dip in form, or the incompetence of a first team coach.

For that reason, it might be far better for clubs to invest in analytics as a safeguard against dumb transfer decisions, rather than as a means to make the ‘perfect’ transfer. After all, the window is open for a limited time. Stuff has to get done. Rather than waste time on a specific bid that will go nowhere, a club director should just be able to ask their analyst “What’s the worst thing that could happen if I get this guy instead of this guy?”, or even “Who should we sell right now?”

5. The most useful metrics are sometimes the ones within plain sight

A lot of ‘amateur’ analysts and journalists think they’re just spinning their wheels publishing their work online while the clubs rely on their own internal information in order to decide who to buy. But it’s equally possible that the best information comes in the simplest packages, and that club analysts read the Internet like the rest of us. I think here of Ted Knutson’s simple and attractive use of assists among U22 players as a good metric for judging attacking midfielders.

As more and more analytical work goes mainstream (thanks as ever to Adam Bate, who is leading the charge over at Sky Sports), more club scouts and analysts might use simple, publicly available analyses in addition to their own work in helping to make transfer recommendations. Big data isn’t always better, and with all the luck and unforeseeable circumstances involved in spending big money on players, it might be better to just go with something as simple as, does a striker shoot the ball to the sides of the net more often than the centre. Perhaps the best club analysts are just trolling Squakwa and James Grayson’s blog like the rest of us. Scary stuff, I know!

6. General data on athletic ability and decay could be far more powerful than assists/goals

One of the more powerful presentations I saw at the Sloan MIT Sports Analytics conference in Boston last March was a paper from a baseball analyst who used an extraordinarily simple yet brilliant method to develop a top prospects list. The list he presented was in many ways identical with the scouts list in publications like Baseball Prospectus, but his came entirely from a forecast model. All he did was to take a huge pile of historical data on prospects, and isolate which key performance indicators extrapolate well from youth (under 18) into an adult career. They weren’t always the ones you might think (ie home runs aren’t the be all and end all at a certain age).

I think this approach has huge potential in a football setting. Are there key performance indicators that demonstrate underlying ability, longevity of skill, and, perhaps more important, susceptibility to repeated injury? This entire approach could be a chimera, but it’s a road worth travelling at least.

7. Big transfers are not always bad transfers

Perhaps it’s the lingering effect of Moneyball, but fans of analytics in sport shouldn’t always assume that big money transfers for single star players are always worse than a small collection of less expensive but promising stars who have yet to reach their maximum potential. In other words, it’s still possible that in Spurs’ case, Soldado, Eriksen, Lamela, and Chriches might not equal Gareth Bale, and that Mesut Ozil could be of more potential value to Arsenal than City’s four signings in Navas, Fernandinho, Jovetic and Negredo (although that would be a stretch).

Each transfer should be taken on its own merits in the context of the team’s pre-existing, empirically demonstrable strengths and weaknesses. Which means Bale still might be the signing of the summer.