Imperial College’s Modelling is Even Worse Than We Thought

When Professor Neil Ferguson and his team at Imperial College London have been challenged on their model’s miserable failure to predict the pandemic death toll in Sweden they have always pushed back saying they didn’t model Sweden, disavowing the work of the team at Uppsala University which adapted their modelling to the Swedish context. But it turns out this is not exactly accurate. Phillip W. Magness explains on AIER:

In the House of Lords hearing from last year, Conservative member Viscount Ridley grilled Ferguson over the Swedish adaptation of his model: “Uppsala University took the Imperial College model – or one of them – and adapted it to Sweden and forecasted deaths in Sweden of over 90,000 by the end of May if there was no lockdown and 40,000 if a full lockdown was enforced.” With such extreme disparities between the projections and reality, how could the Imperial team continue to guide policy through their modelling?

Ferguson snapped back, disavowing any connection to the Swedish results: “First of all, they did not use our model. They developed a model of their own. We had no role in parameterising it. Generally, the key aspect of modelling is how well you parameterise it against the available data. But to be absolutely clear they did not use our model, they didn’t adapt our model.”

The Imperial College modeller offered no evidence that the Uppsala team had erred in their application of his approach. The since-published version from the Uppsala team makes it absolutely clear that they constructed the Swedish adaptation directly from Imperial’s UK model. “We used an individual agent-based model based on the framework published by Ferguson and co-workers that we have reimplemented” for Sweden, the authors explain. They also acknowledged that their modelled projections far exceeded observed outcomes, although they attribute the differences somewhat questionably to voluntary behavioural changes rather than a fault in the model design.

Ferguson’s team has nonetheless aggressively attempted to dissociate itself from the Uppsala adaptation of their work. After the UK Spectator called attention to the Swedish results last spring, Imperial College tweeted out that “Professor Ferguson and the Imperial COVID-19 response team never estimated 40,000 or 100,000 Swedish deaths. Imperial’s work is being conflated with that of an entirely separate group of researchers.” It’s a deflection that Ferguson and his defenders have repeated many times since.

In fact, though, as Phillip points out, it is not true to say that the Imperial team never estimated 40,000 or 100,000 Swedish deaths. Hidden away in a spreadsheet in the appendix to Report 12, published on March 26th 2020, are the team’s estimates for other countries including Sweden. The projections are expressly intended to encourage those countries to follow suit with social restrictions. They write:

To help inform country strategies in the coming weeks, we provide here summary statistics of the potential impact of mitigation and suppression strategies in all countries across the world. These illustrate the need to act early, and the impact that failure to do so is likely to have on local health systems.

The predictions for Sweden are up to 90,157 deaths under “unmitigated” spread (Uppsala projected 96,000) and, under “population-level social distancing” (lockdowns), 42,473 deaths (compared to Uppsala’s 40,000). So, contrary to their repeated denials, Ferguson’s team did make predictions for Sweden very close to those made by the Uppsala team who adapted their model, and those predictions were just as way off. Sweden’s Covid death toll at the end of the first wave, on August 31st, was 5,821.

Phillip summarises further failures of the Imperial modelling in a table showing four non-lockdown countries (Sweden, Taiwan, South Korea, Japan) and the United States (most of whose states imposed a lockdown in the spring) with their one-year death toll and how it compares to Imperial’s projections.

It’s worth saying, though, that the models for the ‘unmitigated’ scenarios predicted the deaths to occur over the course of a few months, not a whole year including another winter flu season. There will be another ‘wave’ of deaths every winter, possibly from (or with) COVID-19 if it remains the dominant respiratory virus (and if we keep on testing for it). If we keep on adding the deaths over several seasons then of course they will eventually reach the predicted figures. But that wasn’t what the models were claiming to show and would be a case of making the evidence fit the model.

The AIER article is worth reading in full.