Archive for September 27th, 2008

I smell something iffy.. or should that be fishy?

September 27, 2008

“I smell something iffy

Quite distinctly fishy

Durham‘s special trial ishy*

When nurr vote got done…”

(With apologies to Alex Glasgow, David Fanshawe, and North-Easterners in general.)

*”Initiative”

In a strange bit of synergistic convergence, the very week that I found myself reading a Ph.D. transfer report dealing with fish oils (of which more later), the Durham Fish Oil Zombie rose from the dead to haunt the national press once more.

The Fish Oil Zombie story (a.k.a. “Fishy Business at Durham Council”) is one of the things that first got me commenting on Ben Goldacre’s Badscience blog over two years ago. I know the story is old, because I started using it as a 1st year student exercise in spotting bad science back in the 2006-7 academic year.

If anyone doesn’t know the history, the place to start is over at Bad Science on the Fish Oil category tab, where you will find the whole history and all the posts. But it is worth pointing out that the Fish Oil Zombie story attracted attention in part because it seemed to epitomize all the things that are bogus about the promotion of unproven nutritional supplements.

Yes, the Fish Oil Zombie story had it all.

To list a few only:

“Pills for Ills”, with shiny capsules as a solution for social problems;

PR spin masquerading as scientific investigation, and “Big Quacka” bullshit;

– the co-opting of public servants (with their apparently enthusiastic connivance) by the clammy embrace of the fish-oil salesmen and the desperate pursuit of column inches;

– a large experiment on a load of kids without anything like “ethical approval”, and indeed without Durham Council ever having formally voted on it;

– the public, and especially the parents and children of Durham, as pawns and dupes;

– a “cover story” from Durham Council’s PR flacks that seemed to change every day

– and – above all – reams of fawningly credulous press coverage.

Ben Goldacre’s Guardian column today nicely summarises both the background and the recent revival. The short version is that, having eventually admitted that their “study” / “initiative” / publicity stunt could in no way be described as a piece of science, Durham Council’s Dave Ford (ex Chief Schools Inspector for Durham and now “Head of Achievement Services”) and Dr Madeleine Portwood (Chief Educational Psychologist and enthusiastic conference attender) have now – eighteen months down the track – magicked up some data.

Ben Goldacre has already pointed out exactly why Durham’s relentlessly massaged figures don’t show anything in their new form, just as they showed nothing when the GCSE results appeared; the “study” “Initiative” trial was incapable of producing reliable data, because it was designed that way from the outset. It was, from Day One, a PR stunt and a piece of let’s-be-seen-to-be-doing-something-ness. And all the post hoc finagling in the world can’t change that – or make a silk purse out of a sow’s ear.

The shorthand for this, in science as in many other walks of life, is:

“Garbage in – garbage out”

Or: if you run a study giving kids fish oil capsules with no control group, you cannot reach any meaningful conclusion on whether the fish oils do anything, regardless of your (claimed) sample size.. This is, as they say, not exactly rocket science.

More interesting than the details of this rather desperate attempt to generate a bit more PR and publicity out of re-animating the Fish Oil Zombie is the timing.

I don’t mean the full moon, or the approach of Halloween. I mean the fact that the Zombie has risen from the grave precisely three weeks before the ”Food for the Brain” conference in London, organised and promoted by fish-oil friend and enthusiast Patrick Holford’s crew… and at which Portwood is scheduled to give the assembled Nutritionistas The Word From Durham.

Oddly unmentioned in the recent stories is that the fish oil capsules in Durham’s trial “Initiative” were provided free by the fish-oil manufacturer Equazen. The FFTB conference is being run by Dear Patrick’s acolytes (FFTB and the supplement company Biocare), but is also supported by Equazen.

Biocare, of course, is the company where Mr FFTB, Patrick Holford, is Head of Science and Education, according to the gang over at Holfordwatch. Biocare sell an extensive range of fish oil products. And Patrick’s face, name and endorsement are on lots of branded fish oil products too.

Equazen’s website front page prominently states:

“EyeQ [Equazen’s fish oils] used in Food For The Brain Schools Campaign” and:

“The Omega Oils used in the Durham Schools Trial”.

Indeed, the line

“Omega-3 fish oils including Eye Q Proven in the Durham Schools Trial”

turns up on Google whenever your search lights upon “equazen.com”.

In other words – it’s a very small fishy old fish-oil world.

Or… evidence, once again, that everything Evil Big Pharma can do – e.g. run “vanity PR” meetings with hand-picked speakers, all designed to act as a surreptitious plug for the company’s drug(s) and message(s) – “Big Quacka” (“Big [Fish] Oil”…?) can, and does, do as well.

There will, of course, be some real scientists speaking at the FFTB meeting – for instance Dr Joe Hibbeln of the US National Institutes of Health. No-one is saying there shouldn’t be fish-oil research. What scientists and skeptics are saying is that there should be good research, done properly, so that we can get some real idea whether fish oils are any good for anything.

Which brings me to my Ph.D. transfer report.

The thesis concerns two things – a smallish human study on the use of fish-oils to reduce cardiac arrhythmias after surgery, and a feeding study in rats to see whether fish oils alter cardiac cell electrical parameters.

This human study, the Ph.D. student told me, was to repeat one that had been done in another country that had shown apparent arrhythmia-reducing effects of the fish oil. He was repeating the study, he said, to check if the effect was real, to check if the effect was as big as the other workers had reported, and to look at whether any clues could be got as to the basis of any effect. This last bit would be done by making sophisticated measurements on ion channels and hormone receptors in small samples of tissue that are routinely taken from the heart during surgery.

As we were discussing the work, I asked the student (who is actually a Cardiology Registrar – a doctor training to become a consultant – taking some years out to do a Ph.D.) about the blinding in the clinical study. The study was double blind, he explained. The oil was given as a capsule, containing either fish oil or olive oil. The patients were asked to swallow it whole, so that they got no taste of the oil in the mouth. As the primary experimenter, he would not know who was in which patient group – who got fish oil and who didn’t – until the study ended after about three years. We talked for a bit about whether there was any way the patients might work out which group they were in (“fishy burps”, for instance – nice), and other issues of the experimental design.

The rat study, too, has the main experimenter (our cardiologist again) blinded as to whether the rat whose cells he is studying had been on the fish oil-supplemented diet, or on a “placebo oil”. Once again, he will not know until the study finishes which animal was which.

Now, these designs are not absolutely perfect, but what they testify to is the desire of scientists to get the answer as free as possible from “confounding effects”. These include “expectation effects” – the tendency of experimenters to see a result when one isn’t there, because they want there to be a positive outcome from the study.

Note, also, the repeating of the earlier clinical study. When something potentially interesting shows up in the scientific / medical literature, the first thing people want to know is – is it real?. So it will get repeated by other investigators in other labs, on other patient populations, in slightly different ways, and so on. It will be confirmed, or sometimes disproved.

Small clinical studies, though, can only tell you so much. They need to be tested on a larger scale. So the next step, if something looks promising in several small trials, will typically be a much bigger study, usually on multiple sites, to look at whether the possible therapy holds up in a bigger patient group, perhaps with more diverse characteristics (age, extent of any disease, diet and so on). Any such bigger study will have to go through extremely extensive ethical approval (is the study and what it asks patients to do justified on the grounds of the benefit likely to be derived? Is it safe?). The big study will also have to have as good a blinding procedure as can be devised, and so on, and so on.

Slow and a bit plodding, perhaps, but all exemplifying the desire to find out the truth about whether the stuff really works. And reflecting, as ever, the desire not to fool oneself:

“The first principle [of scientific integrity] is that you must not fool yourself – and you are the easiest person to fool” – Richard P Feynman in his lecture on Cargo Cult Science

And if we are ever going to get real answers on fish oils, that is the kind of work we need. Proper study design; proper blinding; proper data analysis; professional investigators.

In contrast, what we don’t need is PR-over-spun garbage, designed to polish up peoples’ and companies’ public images but devoid of any scientific usefulness whatsoever.

Ben Goldacre is continuing to campaign, via his Guardian column and blog, for journalists who write about science and health stories to take the fairly elementary steps to be able to spot the difference.

Let’s just hope his message is getting through.

[BPSDB]

Advertisements