What does the Bank of England have in common with Roman priests?

My word of the week is haruspex. A haruspex was a Roman priest trained in the art of predicting the future by inspecting chicken entrails. Before every major decision, Roman emperors would call for their haruspex to slaughter a few chickens and predict the future.

Hopefully, we can agree that inspecting chicken entrails is not likely to deliver much insight. Therefore, we take a more scientific approach today and rely on the services of economists. Alarmingly for the profession, their track record is not much better than the entrail readers. I’ve previously discussed the International Monetary Fund’s (in)ability to anticipate recessions. Today, I’m looking at the Bank of England (BoE)’s prowess as an inflation predictor.

The chart below looks at the relationship between the BoE’s published forecasts and subsequent inflation outturns in the second year of their projections.

There has been much hand-wringing over the failure to anticipate inflation in the wake of the COVID-19 shock. This misses the point. I’ve intentionally excluded 2022 from the chart because the forecast error was so enormous (anticipated inflation: 2.1%, realised inflation: 10.8%). Ignoring that massive outlier, there still is a large standard error and a negative correlation between forecasts and outturns over the past 20 years.

Taken literally, a negative correlation implies you would be better off totally ignoring these forecasts than paying any attention to them. Let’s be generous and assume that is a small sample result. We can still say that the medium-term BoE inflation forecasts have had zero information content since 2004.

Remember that the next time you read media commentary talking in reverent terms about the latest Monetary Policy Report and whether inflation is forecast to be marginally above/below target.

Why so inaccurate?

If one were to view things a little more positively, then (excluding the COVID-19 shock) inflation has on average been close to 2%, so it might be fair to say that the forecast errors have roughly netted out. Furthermore, the two other big overshoots were the result of the sharp rise in oil prices in 2008 and the effects of lagged sterling depreciation and a commodity price rebound in 2011.

In any case, these comments shouldn’t be read as an attack on the institution. There are lots of factors at play here.

First, I’ve focused on the second year of their forecasts. The track record in anticipating the near term is much stronger, partly assisted by base effects. Second, economic forecasts are presented as conditional on certain assumptions, notably about commodity prices and the exchange rate. If those conditioning assumption turn out to be incorrect, the forecasts will be incorrect.

Third, and most important, inflation is endogenous to the BoE’s own actions. If they anticipate high inflation, they should respond with tighter monetary policy, which makes the outcome they anticipate less likely to transpire.

The value in mapping uncertainty

The correct response to this track record is not “the Bank of England is indefensibly bad at forecasting inflation” but “unconditional inflation forecasting is prohibitively difficult”.

As Stanford University’s Paul Saffo points out, forecasting is not the same as haruspicy[1]. In his words, “the primary goal of forecasting is to identify the full range of possibilities, not a limited set of illusory certainties”. He describes the forecasters’ task as “mapping uncertainty” and providing a framework for understanding how changing assumptions impact outcomes.

Understood that way, the forecasters’ role is not to divine the future but to provide an organising framework for understanding the present. In the words of statistician George Box (1919-2013), “all models are wrong, but some are useful”.

As an empirical guide to future inflation, models based on chicken entrails and economic relationships are both wrong. The latter are useful to the extent they allow us to think about the world more clearly.

[1] Haruspicy is the job performed by a haruspex. That should not be confused with augury. Augury was performed by an augur, another type of Roman priest, who divined the future by looking at flocks of birds in flight. When you next hear two economists having an argument about the outlook, imagine the similarly passionate debates between augurs and haruspices that took place 2,000 years ago.