I smell something iffy.. or should that be fishy?

“I smell something iffy

Quite distinctly fishy

Durham‘s special trial ishy*

When nurr vote got done…”

(With apologies to Alex Glasgow, David Fanshawe, and North-Easterners in general.)

*”Initiative”

In a strange bit of synergistic convergence, the very week that I found myself reading a Ph.D. transfer report dealing with fish oils (of which more later), the Durham Fish Oil Zombie rose from the dead to haunt the national press once more.

The Fish Oil Zombie story (a.k.a. “Fishy Business at Durham Council”) is one of the things that first got me commenting on Ben Goldacre’s Badscience blog over two years ago. I know the story is old, because I started using it as a 1st year student exercise in spotting bad science back in the 2006-7 academic year.

If anyone doesn’t know the history, the place to start is over at Bad Science on the Fish Oil category tab, where you will find the whole history and all the posts. But it is worth pointing out that the Fish Oil Zombie story attracted attention in part because it seemed to epitomize all the things that are bogus about the promotion of unproven nutritional supplements.

Yes, the Fish Oil Zombie story had it all.

To list a few only:

- “Pills for Ills”, with shiny capsules as a solution for social problems;

- PR spin masquerading as scientific investigation, and “Big Quacka” bullshit;

- the co-opting of public servants (with their apparently enthusiastic connivance) by the clammy embrace of the fish-oil salesmen and the desperate pursuit of column inches;

- a large experiment on a load of kids without anything like “ethical approval”, and indeed without Durham Council ever having formally voted on it;

- the public, and especially the parents and children of Durham, as pawns and dupes;

- a “cover story” from Durham Council’s PR flacks that seemed to change every day

- and – above all – reams of fawningly credulous press coverage.

Ben Goldacre’s Guardian column today nicely summarises both the background and the recent revival. The short version is that, having eventually admitted that their “study” / “initiative” / publicity stunt could in no way be described as a piece of science, Durham Council’s Dave Ford (ex Chief Schools Inspector for Durham and now “Head of Achievement Services”) and Dr Madeleine Portwood (Chief Educational Psychologist and enthusiastic conference attender) have now – eighteen months down the track – magicked up some data.

Ben Goldacre has already pointed out exactly why Durham’s relentlessly massaged figures don’t show anything in their new form, just as they showed nothing when the GCSE results appeared; the “study” “Initiative” trial was incapable of producing reliable data, because it was designed that way from the outset. It was, from Day One, a PR stunt and a piece of let’s-be-seen-to-be-doing-something-ness. And all the post hoc finagling in the world can’t change that – or make a silk purse out of a sow’s ear.

The shorthand for this, in science as in many other walks of life, is:

“Garbage in – garbage out”

Or: if you run a study giving kids fish oil capsules with no control group, you cannot reach any meaningful conclusion on whether the fish oils do anything, regardless of your (claimed) sample size.. This is, as they say, not exactly rocket science.

More interesting than the details of this rather desperate attempt to generate a bit more PR and publicity out of re-animating the Fish Oil Zombie is the timing.

I don’t mean the full moon, or the approach of Halloween. I mean the fact that the Zombie has risen from the grave precisely three weeks before the ”Food for the Brain” conference in London, organised and promoted by fish-oil friend and enthusiast Patrick Holford’s crew… and at which Portwood is scheduled to give the assembled Nutritionistas The Word From Durham.

Oddly unmentioned in the recent stories is that the fish oil capsules in Durham’s trial “Initiative” were provided free by the fish-oil manufacturer Equazen. The FFTB conference is being run by Dear Patrick’s acolytes (FFTB and the supplement company Biocare), but is also supported by Equazen.

Biocare, of course, is the company where Mr FFTB, Patrick Holford, is Head of Science and Education, according to the gang over at Holfordwatch. Biocare sell an extensive range of fish oil products. And Patrick’s face, name and endorsement are on lots of branded fish oil products too.

Equazen’s website front page prominently states:

“EyeQ [Equazen’s fish oils] used in Food For The Brain Schools Campaign” and:

“The Omega Oils used in the Durham Schools Trial”.

Indeed, the line

“Omega-3 fish oils including Eye Q Proven in the Durham Schools Trial”

turns up on Google whenever your search lights upon “equazen.com”.

In other words – it’s a very small fishy old fish-oil world.

Or… evidence, once again, that everything Evil Big Pharma can do – e.g. run “vanity PR” meetings with hand-picked speakers, all designed to act as a surreptitious plug for the company’s drug(s) and message(s) – “Big Quacka” (“Big [Fish] Oil”…?) can, and does, do as well.

There will, of course, be some real scientists speaking at the FFTB meeting – for instance Dr Joe Hibbeln of the US National Institutes of Health. No-one is saying there shouldn’t be fish-oil research. What scientists and skeptics are saying is that there should be good research, done properly, so that we can get some real idea whether fish oils are any good for anything.

Which brings me to my Ph.D. transfer report.

The thesis concerns two things – a smallish human study on the use of fish-oils to reduce cardiac arrhythmias after surgery, and a feeding study in rats to see whether fish oils alter cardiac cell electrical parameters.

This human study, the Ph.D. student told me, was to repeat one that had been done in another country that had shown apparent arrhythmia-reducing effects of the fish oil. He was repeating the study, he said, to check if the effect was real, to check if the effect was as big as the other workers had reported, and to look at whether any clues could be got as to the basis of any effect. This last bit would be done by making sophisticated measurements on ion channels and hormone receptors in small samples of tissue that are routinely taken from the heart during surgery.

As we were discussing the work, I asked the student (who is actually a Cardiology Registrar – a doctor training to become a consultant – taking some years out to do a Ph.D.) about the blinding in the clinical study. The study was double blind, he explained. The oil was given as a capsule, containing either fish oil or olive oil. The patients were asked to swallow it whole, so that they got no taste of the oil in the mouth. As the primary experimenter, he would not know who was in which patient group – who got fish oil and who didn’t – until the study ended after about three years. We talked for a bit about whether there was any way the patients might work out which group they were in (“fishy burps”, for instance – nice), and other issues of the experimental design.

The rat study, too, has the main experimenter (our cardiologist again) blinded as to whether the rat whose cells he is studying had been on the fish oil-supplemented diet, or on a “placebo oil”. Once again, he will not know until the study finishes which animal was which.

Now, these designs are not absolutely perfect, but what they testify to is the desire of scientists to get the answer as free as possible from “confounding effects”. These include “expectation effects” – the tendency of experimenters to see a result when one isn’t there, because they want there to be a positive outcome from the study.

Note, also, the repeating of the earlier clinical study. When something potentially interesting shows up in the scientific / medical literature, the first thing people want to know is – is it real?. So it will get repeated by other investigators in other labs, on other patient populations, in slightly different ways, and so on. It will be confirmed, or sometimes disproved.

Small clinical studies, though, can only tell you so much. They need to be tested on a larger scale. So the next step, if something looks promising in several small trials, will typically be a much bigger study, usually on multiple sites, to look at whether the possible therapy holds up in a bigger patient group, perhaps with more diverse characteristics (age, extent of any disease, diet and so on). Any such bigger study will have to go through extremely extensive ethical approval (is the study and what it asks patients to do justified on the grounds of the benefit likely to be derived? Is it safe?). The big study will also have to have as good a blinding procedure as can be devised, and so on, and so on.

Slow and a bit plodding, perhaps, but all exemplifying the desire to find out the truth about whether the stuff really works. And reflecting, as ever, the desire not to fool oneself:

“The first principle [of scientific integrity] is that you must not fool yourself – and you are the easiest person to fool” – Richard P Feynman in his lecture on Cargo Cult Science

And if we are ever going to get real answers on fish oils, that is the kind of work we need. Proper study design; proper blinding; proper data analysis; professional investigators.

In contrast, what we don’t need is PR-over-spun garbage, designed to polish up peoples’ and companies’ public images but devoid of any scientific usefulness whatsoever.

Ben Goldacre is continuing to campaign, via his Guardian column and blog, for journalists who write about science and health stories to take the fairly elementary steps to be able to spot the difference.

Let’s just hope his message is getting through.

[BPSDB]

About these ads

19 Responses to “I smell something iffy.. or should that be fishy?”

  1. She-Liger Says:

    Well…
    And why has such question arisen?
    And are good investigations about effectiveness/non-effectiveness of fish oil absent at all in the world? 8-O
    Did nobody carry out them?! Never and nowhere?
    Strangely!…

  2. jonhw Says:

    On a tangent – if you’re able to discuss this on your blog, did you come up with any kind of solution to ‘fishy burps’ letting people know that they’re in the fish oil (not placebo) group? I was wondering whether people might use some kind of fishy placebo?

  3. BobP Says:

    If Portwood is being paid for her contribution to the “Food for the Brain” shebang, she will need to declare a conflict of interest to her employers

  4. draust Says:

    did you come up with any kind of solution to ‘fishy burps’ letting people know that they’re in the fish oil (not placebo) group? – JonHW

    We didn’t really discuss this – I think all he was going to do was record the number of patients who specifically mentioned GI side effects (inc. fishy burps!) and then report it for both fish oil and the placebo (olive oil) groups.

    Ideally you would want a fishy oil that wasn’t rich in omega-3s, I suppose. I don’t know whether such actually exists, though – perhaps some other commentator can tell us. It’s not really my field – I was reading the report because of my background in cell physiology and cardiac physiology, rather than for my knowledge of fish oil.

    If Portwood is being paid for her contribution to the “Food for the Brain” shebang, she will need to declare a conflict of interest to her employers – BobP

    Agreed, Bob, but I suspect FFTB would only be paying her expenses (travel and accommodation). That would be typical for academic conference speakers.

    It would be interesting to know if there was any honorarium involved for the FFTB event. Over in mainstream science/medicine, one of the things that distinguishes a Pharma “PR / vanity” conference from a bona fide one is often that the speakers in the vanity events are being paid a significant appearance fee, either overt or disguised.

    Of course, it is not always totally clear cut – the old joke about this is the one of the eminent clinician being invited to a 3-day event in a posh Alpine ski resort, first class air-fare, 4-star ski lodge and free ski hire and ski pass, and only having to talk for 45 minutes on one day. No formal fee, but..

    Of course, this is an apocryphal example, and is one of those “people say this used to go on” rather than something I (or any of my mates) have experienced personally. The grand total of all that I personally have ever got off Big Pharma over a quarter century in the biz is one free lunch at the AstraZeneca Alderley Park restaurant.

    Talking of honoraria, I would be interested to know if Equa£en made any “expenses” payments to Durham CC or its employees during the Fish Oil trial “initiative”. Obviously Equa£en donated the capsules free, but I wonder if they were financially underwriting anything else, or whether it was otherwise all on the ratepayers’ tab. I can’t remember if any of the FOI campaigners or Durham sleuths, like MacCruiskeen, have ever asked this question, although they probably have.

  5. jonhw Says:

    Thanks DrAust. I’ve seen a FOIA response where Durham says – effectively – that they got no money from Equazen (though clearly they did get the pills free).

  6. Claire Says:

    “No formal fee but…” [Dr Aust]

    free raffle tickets?”

  7. maccruiskeen Says:

    David Ford wrote to me on 13th November 2006 and stated, “There have been no financial inducements from Equazen in cash or kind.”

    He was also asked asked about inducements by reporters from BBC “You and Yours” and strenuously denied that any had been offered or taken.

    I believe him.

  8. jdc325 Says:

    Re making fishy burps – is there something edible and fishy-smelling that can be used to make placebo capsules suitably odourous? I have a vague idea something like methyl or ethyl amine may do the trick. Diisopropyl amine, maybe? I’m pretty sure these smell fishy, but I’m not so sure whether they are edible!

    Re EPA/DHA-free fish oil – apparently the Equazen process uses winterisation and molecular distillation to increase the relative amounts of Eicosapentaenoic Acid and Docosahexaenoic Acid. I assume that it would also be possible to manipulate the levels downwards. I guess if you started with a fish oil that was relatively low in EPA/DHA it would be easier to achieve a low EPA/DHA level. E.g., cod contains @150mg EPA/DHA per 100g whereas sardine contains about 1000mg per 100g [USDA].

    PDF of the ‘Equazen Process’

  9. Kat Says:

    Claire suggested free raffle tickets. Given the prizes, wouldn’t you want to give the tickets back again?

    TAKE A LOOK AT THESE FABULOUS PRIZES TO BE WON…

    Consultation at the Brain Bio Clinic
    Fresh Water 1000 filter system worth £250
    YorkTest Food Allergy Kit worth £250
    2 Wholefood’s Market Vouchers, £150 each
    Bio Care Voucher worth £100
    Equazen Voucher worth £100
    2 Doves Spa vouchers – Spa centres around UK
    3 YorkTest Homocysteine Kits £75 each
    2 100% Health 2 year membership worth £50 each
    6 of Patrick’s latest books ‘How to Quit + Food Glorious Food’ worth £30
    Basket of Environ Skin Care products

  10. draust Says:

    Ha ha ha….. nice spot, Kat.

    It does have a touch of the busman’s holiday feel, though that isn’t the right idiom. Do you think that the lower you finish in the raffle, the more of Patrick’s books you get to take home?

    Re. jdc’s coments about the oil and what would be a better placebo, the Holfordwatch gang suggested “tinned tuna oil”, or similar, tinned tuna having pretty minimal omega-3 levels.

    The wider point is, I guess, that if you were running a major, randomized, blinded human placebo trial of your fish oil, then having a proper “active placebo” would be a large part of making the experiments reliable. I wonder if the Prof John Stein / Wellcome Trust incarcerated young offender fish oil trial is going to use a more sophisticated placebo than (say) olive oil? I would hope so. The news stories said “placebo trial” but didn’t mention what the placebo was going to be – though I would think that kind of info would have to have been in the original grant application and the Ethical Approval submissions, inter alia.

    Of course, Durham avoided this irksome sort of stuff by having no control group, let alone no placebo arm.

  11. Svetlana Perstovich Says:

    I see that you hung about DC’s blog today… The post about A.J.Clark attracted you.
    OK!
    I want to say to all of you, who are blabbing here, about A.J.Clark!
    A.J. Clark DIDN’T collaborate with quacks!!!!! He NEVER worked in quackery organization (sort of OfQuack!!!) !!!!! He was a HEAD OF PHARMACOLOGY CHAIR!!!!!! And he NEVER hid his honest name under pseudonyms!!! Never! – even when quacks sued him and he lost a case!!! He HAD ORGANIZED his Chair, and he NEVER buried Chairs, where he worked and NEVER wrote obituaries in memory of those Chairs!!!

    Have you understood me??!!!!

    :( :( :(

  12. Nash Says:

    Is promoting Omega-3 fish oil eco friendly? Our oceans are already being over fished.

  13. Dr Aust Says:

    If one were talking about actually eating oily fish, then I guess the precise “eco-friendliness” would depend on what fish from what source.

    Not sure about fish oils. They seem to be processed and enriched in some way, so it sounds like there is a lot of “production processing”, which would be energy-costly. Of course, a counter-argument might be that they can use bits of the fish that in normal eating would be thrown away. Equa$en do give some info on the process on their website, but I can’t be bother to go and look. Perhaps someone else can check it out.

    On the whole, though, I would imagine that promoting the actual eating of oily fish would be more eco-friendly. But then, “Pills for Ills” seems to be the modern way. I remember one of our neighbourhood middle-class mothers yakking on in the playground about how she was stuffing her two kids with fish oil capsules. “Have you tried giving them fish?” I asked. “Oh, they won’t touch fish”, she said. Hmm… now whose fault is that? Children learn to like what their parents present to them as good to eat, in my experience.

  14. Stephen Brown Says:

    About the flavour of the placebo capsules (obviously not used in the Durham Year 11 trial/initiative/study or whatever we should call it.)

    A previous trial (written up by Richardson http://pediatrics.aappublications.org/cgi/reprint/115/5/1360 ) quotes “Placebo treatment consisted of olive oil capsules,which were carefully matched with the active treatment with respect to both appearance and flavor.”

    Note the last word ‘flavor’. As I am sure Dr Richardson would be accurate – she has published often on EFA research – Presumably something must have been included in the placebo to give the ‘fishy taste’

    I note this was done in County Durham so presumably the officers know how to do RCTs and were, presumably, sufficently impressed by the results they saw in the children to consider offering the supplement to all Year 11 pupils.

    Anyone going to the FTTB conference? I am sure Dr Portwood would enjoy answering questions on the trial!

  15. Grace Gockley Says:

    Hey neat website, just want to ask you what antispam software you have on your site for comments because I am getting so many spammers on my blog.

  16. anetad Says:

    There’s a bad fame attached to this specific omega 3 supplementation. People should remember that the attempt and the method of the trial may not be good, but the supplement itself may provide benefits.
    Before giving inconsiderate statements we need to find out more about benefits of good brands of omega 3 fish oil supplements and the detailed research behind it.

  17. draust Says:

    Goodness. The comment spammers are getting more and more ingenious.

    That last one from “anetad” links to a site flogging Omega-3 supplements. I should have spammed it but I’ve let it through just to serve as a warning to other bloggers of how much the comments spammers are prepared to write to try and fool you and/or the auto spam filtering.

  18. anetad Says:

    I do not intend to fool anyone. I have noticed that bad evaluation of something has a great influence on people (especially lazy, who believe in everything they read). That’s why I suggest to refer to related research. I’d like people to be aware of recent findings…I do not try to flog anything.

  19. draust Says:

    It was your name linking directly to a marketing site from a company (Vegepa) selling Omega-3 supplements that led me to identify it as comment-spam, Anetad. The reason is that this is a typical tactic of on-line spammers – something that looks like a short comment (usually just a phrase or a line), where the commenter’s name links straight to a promotional or sales site. Most bloggers get multiple such “pseudo-comments” every day.

    If you want to avoid being labelled that way, best to write something like your second comment and then say:

    “Personally I find the information on this company’s site (linked) quite convincing.”

    That way people know you are highlighting a link to a commercial site which offers info about Omega-3′s, and they can choose whether to go and take a look. Or not.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Follow

Get every new post delivered to your Inbox.

Join 1,991 other followers

%d bloggers like this: