The problem with cost-benefit analysis
OR people and economists often use cost-benefit analysis (CBA) to help advise decision-makers (DMs). And in many cases it is a useful tool. When the DM has a number of choices, and the outcomes are multi-dimensional and include some qualitative results, it is often helpful to try and assign costs to try and reduce the number of dimensions that the DM needs to look at.
So, at the outset, I give CBA thumbs up, some of the time. But not all the time.
And the problem with CBA is in those three words, cost, benefit & analysis.
Measuring the cost of a decision and its outcome is a process that is fraught with pitfalls. From time to time, the case of Stewkley church is mentioned. When a new London airport was being considered, one site that was seriously considered was near Stewkley. The church there would have to be demolished to make way for the proposed runways. The study for the government valued the church at the insurance value, as it was insured against total destruction by fire. But the church dated from about 1180, with the site having been used for at least 100 years before that. It was an irreplaceable piece of architecture, a part of the local landscape. Someone suggested, seriously, but with a mischievous glint in his eye, that the proper value should be the construction cost, say £100, extrapolated to the present day, roughly 800 years, at 10% per year. That puts a value of £1.3e+35 on the building. One costing makes the church of negligible value compared with the cost of the airport, the other makes the airport of negligible value compared with the church. Although this is an extreme example, it is very hard to put a cost on anything which is rare or unique, or which has aesthetic value. And, sometimes proponents of CBA overlook their philosophy that "one measures what is convenient to measure".
What about "benefit"? All OR models and economic models are simplifications of reality. So the benefits that are shown by the models are only a simplification of what would happen as a result of the decisions. I have mentioned in earlier blogs that there is a problem in defining the "system" when considering the effect of a decision, or the scope of a model. As a result, the benefits in CBA are often limited to the ones that the modeller wishes to include, and the resulting scenario is poor in detail. Society and the technology we use is changing rapidly, so talking of future benefits may simply extrapolate the benefits that are currently experienced.
And "analysis"? Well. CBA is performed for clients. Clients have their prejudices, and - shame! - sometimes the process of CBA can be biased towards those prejudices.
When I was a teenager, my father and I discussed replacing an old camera with a new, better one. I would have needed an advance on my pocket money, I think. So, I set out the cost of taking pictures with the camera that I then used, compared with the cost of taking the same number using a new one, and I manipulated the figures in my favour by showing that after about 150 pictures (on Kodachrome colour film) I would be better off with the new one. And so I got the camera. Had I known today's collector's value on the old camera, I should have kept it as well. But I recall that experience for demonstrating how one could model decisions, and possibly bias the results. My dad knew that I was manipulating the data but he went along with it, because he knew that I would need a better camera sooner or later.
Why mention CBA now? Two events have prompted me to think about it. The first has been the death of Doug Engelbart who was best known for his work in human and computer interaction, leading to the invention of the computer mouse. Had one been writing about the future of computers 40 years ago, and performing CBA on decisions involving computer installations, the proliferation of personal computers, in large part due to computer mice, would not have been foreseen. Teletype terminals - yes - personal computers - no. Teletypes would have extrapolated the present into the future. And the second is the decision by the UK government to take one benefit out of their model for a new railway line, the HS2. Up to the past week, one of the benefits that has been costed has been the value to workers of time saved by having a faster train, Now that benefit has been removed, because so many people work while on a train, using laptops, tablets and phones. Saving 20 to 30 minutes on a journey does not change their productivity by as much as had been calculated earlier. The definition of benefits has changed.
So, at the outset, I give CBA thumbs up, some of the time. But not all the time.
And the problem with CBA is in those three words, cost, benefit & analysis.
Measuring the cost of a decision and its outcome is a process that is fraught with pitfalls. From time to time, the case of Stewkley church is mentioned. When a new London airport was being considered, one site that was seriously considered was near Stewkley. The church there would have to be demolished to make way for the proposed runways. The study for the government valued the church at the insurance value, as it was insured against total destruction by fire. But the church dated from about 1180, with the site having been used for at least 100 years before that. It was an irreplaceable piece of architecture, a part of the local landscape. Someone suggested, seriously, but with a mischievous glint in his eye, that the proper value should be the construction cost, say £100, extrapolated to the present day, roughly 800 years, at 10% per year. That puts a value of £1.3e+35 on the building. One costing makes the church of negligible value compared with the cost of the airport, the other makes the airport of negligible value compared with the church. Although this is an extreme example, it is very hard to put a cost on anything which is rare or unique, or which has aesthetic value. And, sometimes proponents of CBA overlook their philosophy that "one measures what is convenient to measure".
What about "benefit"? All OR models and economic models are simplifications of reality. So the benefits that are shown by the models are only a simplification of what would happen as a result of the decisions. I have mentioned in earlier blogs that there is a problem in defining the "system" when considering the effect of a decision, or the scope of a model. As a result, the benefits in CBA are often limited to the ones that the modeller wishes to include, and the resulting scenario is poor in detail. Society and the technology we use is changing rapidly, so talking of future benefits may simply extrapolate the benefits that are currently experienced.
And "analysis"? Well. CBA is performed for clients. Clients have their prejudices, and - shame! - sometimes the process of CBA can be biased towards those prejudices.
When I was a teenager, my father and I discussed replacing an old camera with a new, better one. I would have needed an advance on my pocket money, I think. So, I set out the cost of taking pictures with the camera that I then used, compared with the cost of taking the same number using a new one, and I manipulated the figures in my favour by showing that after about 150 pictures (on Kodachrome colour film) I would be better off with the new one. And so I got the camera. Had I known today's collector's value on the old camera, I should have kept it as well. But I recall that experience for demonstrating how one could model decisions, and possibly bias the results. My dad knew that I was manipulating the data but he went along with it, because he knew that I would need a better camera sooner or later.
Why mention CBA now? Two events have prompted me to think about it. The first has been the death of Doug Engelbart who was best known for his work in human and computer interaction, leading to the invention of the computer mouse. Had one been writing about the future of computers 40 years ago, and performing CBA on decisions involving computer installations, the proliferation of personal computers, in large part due to computer mice, would not have been foreseen. Teletype terminals - yes - personal computers - no. Teletypes would have extrapolated the present into the future. And the second is the decision by the UK government to take one benefit out of their model for a new railway line, the HS2. Up to the past week, one of the benefits that has been costed has been the value to workers of time saved by having a faster train, Now that benefit has been removed, because so many people work while on a train, using laptops, tablets and phones. Saving 20 to 30 minutes on a journey does not change their productivity by as much as had been calculated earlier. The definition of benefits has changed.
Comments
Post a Comment