If you haven’t had a heart attack, you may now unknot your panties.

Last week, the almighty JAMA published the latest meta-analysis on fish-oil supplementation and secondary prevention of cardiac and stroke events. It’s all the buzz right now, so why shouldn’t I throw in on it?

This hoopla is not unlike the outbreak of “correlation not causation” crap that came out of the Red Meat Will Kill you affair. I wrote about it here. And here. It was much ado about nothing. However, very much like that time, a lot of people have their panties in a knot about this one (mostly proponents of fish oils, mind you).

So what happened here?

Rizos EC et al. Association between Omega-3 fatty acid supplementation and risk of major cardiovascular disease events. JAMA, 308(10):1024, 2012.

When it comes to fish oils, they’re the new panacea. But despite a ba-jillion studies, we still don’t have a definitive answer when it comes to whether they have a benefit towards preventing cardiovascular events like heart attacks and stroke and death. This isn’t a new problem, and in some cases (eg. the use of corticosteroids in premature birth to stimulate the production of lung surfactant), certain therapies did not come to wide-spread use without the pooling of several studies together to show a definitive effect.

I’m not going to go into the nitty gritty of systematic review and meta-analysis methodology in this post, but I would like address some of the criticisms of the study.

Methods:

The authors searched the major scientific medical databases for studies on fish-oil supplementation in adults. Of these studies, they were only interested in randomized controlled trials that were examining the effect of fish oils on cardiovascular events or death. Studies that used supplementation protocols of less than one year were excluded as it was felt that anything less than one year was not enough time on fish oils to produce an effect.

The eligible studies were then assessed for methodological quality. They also looked at whether it made a difference in the conclusion if they pooled both the trials that were blinded vs. not-blinded.

Ultimately, the researchers were interested in whether fish oil supplementation reduced the rate of cardiovascular events or if it reduced the rate of death when compared to a placebo or another diet.

Results:

The initial publication search yielded 3635 studies on fish oils. The meta-analysis included only 20 of this studies. When you examine the flowchart of how studies became excluded, you see that in the first screening,  well over 1000 of them were non-randomized studies. More than 800 of them did not study death or cardiovascular events. And about 1150 of them were review papers, or irrelevant reports (i.e. not even actual experiments). 

Of the 250 studies that were assessed in detail for inclusion, 78 did not study death or cardiovascular events. 50 were non-randomized studies, 42 were reviews or other smaller meta-analyses, and 60 were either duplicate publications or irrelevant reports (i.e. not experimental papers)

[This is a very important part of the paper and unmasks those critics who either didn’t actually read the paper or didn’t have enough understanding of how systematic reviews work. One of the most common criticisms I noted in the news media was that the authors of this study ONLY included 20 studies out of 3635, and that somehow the truth must lie in those 3615 studies that WEREN’T included. Once you read Figure 1 (which I’ve summarized above), it becomes abundantly clear that there is a lot of irrelevant stuff out there and an incredibly large number of non-randomized studies. There’s no point in including studies that didn’t look at cardiovascular events or death, because that’s not what these authors are interested in. You can’t pool non-randomized studies with randomized studies in a meta-analysis (any novice knows that). And review articles and irrelevant reports aren’t experimental evidence. It’s like saying you don’t have the best football team because you didn’t let everyone who showed up for try-outs on the team.

So what evidence, exactly, I would have to ask, is missing here?]

Of the 20 studies included, 2 studies looked at fish-oil “supplementation” through dietary counselling. That is, the intervention studied was educating subjects on how to increase their consumption of fish oils by diet alone, as opposed to capsule supplements. Both studies were not blinded and performed by the same research group. Pooling the studies together did not make sense to the researchers and so these two studies were left out.

Of the 18 remaining studies, 17 looked at fish oils and death (or all-cause mortality), 13 studies looked at fish oils and heart attacks, and 9 studies looked at fish oils and stroke. Four studies were considered studies on mixed prevention (i.e. looking at both subjects who had never had a cardiovascular event and subjects who had previously already had one), 10 studies were on secondary prevention (i.e. looking only at subjects who had already had a previous cardiovascular event), and 4 studies looking at subjects who had implanted defibrillators (which are implanted for a variety of reasons.)

In the cases of fish-oils and death prevention, heart attacks and stroke , there was no evidence to suggest that fish-oils help people who have already had a cardiovascular event live longer, avoid another heart attack, or avoid another stroke.

[Again, one major common criticism in the news media has been that these subjects would all have been on a bunch of medications since they already had had a heart attack or stroke. After a heart attack, you basically buy yourself about 5 medications. While they don’t ALL have definitive proof of decreasing the risk of death or a subsequent heart attack, at least 2 of them do (beta-blockers and aspirin; though aspirin may be on the wane in the next few years,) and the others have more proof than fish-oils. I don’t think it would be a responsible clinical decision making to suggest that fish oils, which, at the time of many of these trials, were yet unproven, take the place of medications that already had proven benefit. Yeah, it’s a tricky ethics question, but the fact remains that fish oils don’t seem to provide any ADDITIONAL benefit over the existing regimen of medications.]

Overall, after pooling all 18 studies, which encompassed the population of a small town (63 279 subjects), the absolute risk reduction was -0.004, with a 95%CI of -0.01 to 0.02, which crosses 0, meaning 0 is a plausible value for the true risk reduction number (since 63 000 people is STILL just a sample.)

[For argument’s sake, let’s say the true risk reduction is spot on -0.004. That means you would have to give 250 people fish oils to prevent one death. With more studies, the confidence interval is only going to get narrower, but it’s not going to change the possibly-irrelevant risk reduction much. The fact that the authors failed to detect a statistically significant p-value associated with this risk reduction doesn’t really signify that we’re just “underpowered”. A practically irrelevant risk reduction is irrelevant, regardless of whether it is statistically significant or not. Just to give an example, some studies estimate that only 17 people need to be treated with beta-blockers to prevent one death.

Some people argue that this is a “trend” towards risk reduction. If I haven’t written about how much this phrase pisses statisticians off, I’ll get around to it eventually. However, by the same argument, if you look at the pooled estimate for stroke risk, there was a “trend” towards an INCREASED risk of stroke while on fish oils. So if we’re going to push things based on “trends” towards statistical significance, there’s a lot more stuff backed up in that pipe.]

Discussion

A lot of lay-press comments have surrounded other issues like the dosage of omega-3’s in the studies, the quality/freshness of omega-3’s in the study, whether or not subjects were compliant in taking their fish oils every day or not, and the list goes on.

I think it’s very important to distinguish bias and methodological error from generalizability.

Generalizability is the extent to which a study can be used to justify decisions for other people. If you read a study on rats, you can’t use those results to justify that treatment for humans. If you do a study with children, you can’t use those results to justify that treatment on adults. In this case, this meta-analysis studied people who had already had a heart attack or stroke. These people were predominantly European (by a lot) with some Japanese. The mean dose of omega-3’s was 1.51g per day.

These characteristics are not flaws of the meta-analysis. It’s not bias. Bias would be if the authors went out and selected studies with only Europeans in them, or if they limited the doses to “lower” ones. This is, in fact, as about unbiased as it gets.

What this DOES mean though, is that if you’re using fish oils and you haven’t had a heart attack or a stroke, you can go right on taking them because no one has figured out whether they have a proven benefit FOR YOU. We can all agree that people who have had a heart attack or stroke are not the same as people who have NOT had a heart attack or stroke. This study does not apply to you. Move along, nothing to see here, folks.

If, however, you’re taking fish oils and you HAVE had a heart attack or stroke, AND you’re of European descent (and maybe if you’re Japanese) AND you’re taking fish oils to prevent ANOTHER heart attack, or stroke or you’re just hoping to live longer, AND you’re taking about 1.51g per day, THEN AND ONLY THEN, should you reconsider your decision to take them, because this study DOES apply to you.

The bottom line: For most of you, you may now unknot your panties (if you had them in a knot before over this.)