Post by robertisaacs on Nov 21, 2008 14:19:38 GMT
Anyone who knows me (especially those who went to Tamworth 1) will know of my fascination with the psychology of science, the way our apparently analytical minds can be tricked so easily.
I therefore found this, the latest from Dr Ben Goldacre, absolutly fascination. I hope you will too.
Before you read it consider this.
If there was a disease which affected one in 1,000,000 people and by taking a drug every single day for the restof your life you could reduce your risk of getting it would you do so?
I therefore found this, the latest from Dr Ben Goldacre, absolutly fascination. I hope you will too.
Before you read it consider this.
If there was a disease which affected one in 1,000,000 people and by taking a drug every single day for the restof your life you could reduce your risk of getting it would you do so?
November 15th, 2008 by Ben Goldacre in presenting numbers, adverts, statistics, bad science |
We’re all suckers for a big number, and you’ll be delighted to hear that the Journal of Consumer Research has huge teams of scientists all eagerly writing up their sinister research on how to exploit us.
One excellent study this month looked at how people choose a digital camera. This will become relevant in three paragraphs’ time. The researchers took a single image, then processed it in Photoshop to make two copies: one where the colours were more vivid, and one where the image was sharper. They told participants that each image came from a different camera, and asked which they wanted to buy. About a quarter chose the one with the more colourful sharper image.
Then the researchers started to pile it on. Firstly they said that this camera had more pixels, using a figure derived from the diagonal width of the sensor: suddenly more than half picked it instead. Then, crucially, they told them that this camera had more pixels, but this time, they used the number of pixels as evidence: a figure measured, as you know, in millions. Suddenly, three quarters chose the supposedly better camera. Just a bigger number. Nothing more.
This week you’ll have noticed the news on rosuvastatin (or Crestor, since either through ignorance, or corporate whoredom, the media love to help drug companies by using their corporate brand names instead of the generic). The JUPITER trial on rosuvastatin has just reported, several months early, and most papers called it a “wonder drug”. The Express, bless them, thought it was an entirely new drug.
“Heart attacks were cut by 54 per cent, strokes by 48 per cent and the need for angioplasty or bypass by 46 per cent among the group on Crestor compared to those taking a placebo or dummy pill”, said the Daily Mail. Dramatic stuff. And in the Guardian, we said: “Researchers found that in the group taking the drug, heart attack risk was down by 54% and stroke by 48%”.
Is this true? Yes. Those are the figures on risk, expressed as something called the “Relative Risk Reduction“. It is the biggest possible number for expressing the change in risk. But 54% lower than what? This was a trial looking at whether it is worth taking a statin if you are at low risk of a heart attack (or a stroke), as a preventive measure: it is a huge market – normal people – but these are also people whose baseline risk is already very low.
If you express the exact same risks from the same trial as an “Absolute Risk Reduction“, suddenly they look a bit less exciting. On placebo, your risk of a heart attack in the trial was 0.37 events per 100 person years, and if you were taking rosuvastatin, it fell to 0.17 events per 100 person years. 0.37 to 0.17. Woohoo. And you have to take a pill every day. And it might have side effects.
And if you express the risk as “Numbers Needed To Treat“, probably the most intuitive and concrete way of expressing a benefit from an intervention, then I reckon, from the back of this envelope in front of me (they naughtily don’t even give the figure in the research paper), that a couple of hundred people need to take the pill to save one life.
Is it a good idea for you personally to take rosuvastatin? That’s not my job here - get over yourself, we’re allowed to talk about ideas - but the way figures are presented can have a huge impact on decisions everyone makes, and this is not idle speculation. In fact the phenomenon has been carefully studied, in many groups, and for many years.
In 1993 Malenka et al recruited 470 patients in a waiting room, and gave them details of a hypothetical disease, and a choice of two hypothetical treatments. In fact it was the same treatment, with the risk expressed in two different ways. 56.8% chose the medication whose benefit was expressed as a relative risk reduction, while only 14.7% chose the medication whose benefit was in absolute terms (15.5% were indifferent).
Are patients uniquely stupid? Joy, no. In fact the exact same result has been found repeatedly in experiments looking at doctors’ prescribing decisions, and even the purchasing decisions of health authorities.
We all love big numbers, and we’re all fooled by big numbers, because we’re all idiots. That’s why it’s important to think clearly, and ignore all newspapers.
We’re all suckers for a big number, and you’ll be delighted to hear that the Journal of Consumer Research has huge teams of scientists all eagerly writing up their sinister research on how to exploit us.
One excellent study this month looked at how people choose a digital camera. This will become relevant in three paragraphs’ time. The researchers took a single image, then processed it in Photoshop to make two copies: one where the colours were more vivid, and one where the image was sharper. They told participants that each image came from a different camera, and asked which they wanted to buy. About a quarter chose the one with the more colourful sharper image.
Then the researchers started to pile it on. Firstly they said that this camera had more pixels, using a figure derived from the diagonal width of the sensor: suddenly more than half picked it instead. Then, crucially, they told them that this camera had more pixels, but this time, they used the number of pixels as evidence: a figure measured, as you know, in millions. Suddenly, three quarters chose the supposedly better camera. Just a bigger number. Nothing more.
This week you’ll have noticed the news on rosuvastatin (or Crestor, since either through ignorance, or corporate whoredom, the media love to help drug companies by using their corporate brand names instead of the generic). The JUPITER trial on rosuvastatin has just reported, several months early, and most papers called it a “wonder drug”. The Express, bless them, thought it was an entirely new drug.
“Heart attacks were cut by 54 per cent, strokes by 48 per cent and the need for angioplasty or bypass by 46 per cent among the group on Crestor compared to those taking a placebo or dummy pill”, said the Daily Mail. Dramatic stuff. And in the Guardian, we said: “Researchers found that in the group taking the drug, heart attack risk was down by 54% and stroke by 48%”.
Is this true? Yes. Those are the figures on risk, expressed as something called the “Relative Risk Reduction“. It is the biggest possible number for expressing the change in risk. But 54% lower than what? This was a trial looking at whether it is worth taking a statin if you are at low risk of a heart attack (or a stroke), as a preventive measure: it is a huge market – normal people – but these are also people whose baseline risk is already very low.
If you express the exact same risks from the same trial as an “Absolute Risk Reduction“, suddenly they look a bit less exciting. On placebo, your risk of a heart attack in the trial was 0.37 events per 100 person years, and if you were taking rosuvastatin, it fell to 0.17 events per 100 person years. 0.37 to 0.17. Woohoo. And you have to take a pill every day. And it might have side effects.
And if you express the risk as “Numbers Needed To Treat“, probably the most intuitive and concrete way of expressing a benefit from an intervention, then I reckon, from the back of this envelope in front of me (they naughtily don’t even give the figure in the research paper), that a couple of hundred people need to take the pill to save one life.
Is it a good idea for you personally to take rosuvastatin? That’s not my job here - get over yourself, we’re allowed to talk about ideas - but the way figures are presented can have a huge impact on decisions everyone makes, and this is not idle speculation. In fact the phenomenon has been carefully studied, in many groups, and for many years.
In 1993 Malenka et al recruited 470 patients in a waiting room, and gave them details of a hypothetical disease, and a choice of two hypothetical treatments. In fact it was the same treatment, with the risk expressed in two different ways. 56.8% chose the medication whose benefit was expressed as a relative risk reduction, while only 14.7% chose the medication whose benefit was in absolute terms (15.5% were indifferent).
Are patients uniquely stupid? Joy, no. In fact the exact same result has been found repeatedly in experiments looking at doctors’ prescribing decisions, and even the purchasing decisions of health authorities.
We all love big numbers, and we’re all fooled by big numbers, because we’re all idiots. That’s why it’s important to think clearly, and ignore all newspapers.