How Not to do a Technology Assessment

In the future Dom Perignon will cost less than drinking Budweiser! Yes, you read that right. I have performed a sophisticated assessment and determined that in the future, drinking champagne need not cost any more than drinking a mass produced beer.

What methodology did I use to arrive at this fantatstic result?

Well, in my analysis I assumed that you will be drinking 1 milliliter (less than a small thimbleful) of the bubbly, as compared to a pint of beer. Thus, by varying the amounts delivered I arrived at the conclusion that the costs of the Dom Perignon will be just a bit smaller than the Budweiser. Drinkers rejoice!

How might you reply? Apples and oranges? Sleight of hand?

Both allegations would be true. If the point is to compare costs of two quantities, it would be methodologically suspect to rearrange variables seemingly arbitrarily in order to equalize costs. It would be far more transparent to present cost per unit, where the units are equal. In this case, the champagne costs about 1,000% more than the beer. Seems like an obvious enough point, no?

However, in an assessment of the costs of various energy technologies released last week, the British government committed the Dom Perignon error. The assessment looked at 4 scenarios, business-as-usual plus 3 alternatives. But the total energy consumed is different across the scenarios. 

The FT caught this methodological problem right away:

[W]hy [is] the “higher renewables” scenario is combined with “more energy efficiency” when the other options are not to the same degree?

(DECC presumes energy saving of 54 per cent in the renewable scenario; 43 per cent under the fossil fuels/biomass scenario and only 31 per cent in the nuclear scenario).

Why is there a presumption that if we move towards wind/solar power then people will use more insulation? It does not necessarily follow. Presumably if you combined the extra energy efficiency to nuclear or fossil fuels they may no longer come out as the most expensive options.

The Guardian, however, did not catch the problem and thus misreported the conclusions:

Every person in Britain will need to pay about £5,000 a year between now and 2050 on rebuilding and using the nation's entire energy system, according to government figures. But the cost of developing clean and sustainable electricity, heating and transport will be very similar to replacing today's ageing and polluting power stations, the analysis finds. . .

The predictions challenge suggestions that the costs of embracing low-carbon energy and meeting the UK's legally binding commitments to tackle global warming will be higher than the bill would be for using traditional energy sources.

The actual apples-to-apples math is pretty straightforward (under the assumptions of the analysis) -- if the costs of delivering about half the supply renewable energy are about the same as the costs of delivering non-renewable energy, then the renewable energy costs twice as much per unit of energy delivered.

No one is well served by claiming that Dom Perignon costs the same as Budweiser, because it doesn't.

Posted in: