HMB-FA: The post-mortem addendum

Post-mortem edit (April 12, 2014):

The first author of this paper posted a video to counter some of the points made by another website criticizing his study. Here’s what I gleaned from it:

1) The author states that they screened around 100 people before they _chose_ the 24 subjects in the study. There’s nothing wrong with pre-screening. Assessing eligibility is a vital part of any randomized trial. What’s wrong is that this isn’t reported; nor is the “extra” inclusion criteria. If you screen 100 people, and the first 25 meet your pre-planned inclusion criteria, then those are the 25 you have to take. If you then _added_ inclusion criteria (like, “had to be mentally tough enough to go through a training camp”; however you measure that–was there a run-in period?) then you report that. This speaks volumes about generalizability and the lack thereof in this study. This is also why the CONSORT statement has a flow diagram. It’s not hard to do, but it allows readers to make decisions on how to interpret the data.

With this NEW information, the study seems less and less relevant. If only, “high responders to training stimulus” were selected, how were these “responders” identified as such? This also changes the research question completely to, “What is the effect of HMB-FA and a periodized daily undulating resistance program with 2 weeks of overreaching on strength in high responders to training stimulus?” (without a clear definition of what ‘high responders’ actually means) and, in fact makes it quite difficult to assess whether the daily undulation is actually a vital part of this since there’s no group that wasn’t undulating on HMB. However, the main point I’m trying to make here is that if these subjects were picked for their ‘inborn traits’ (both physical and mental), then the reported inclusion criteria were actually NOT the inclusion criteria, and we’re looking at a rapidly diminishing relevance to the point of the study being a great academic paper on which to generate hypotheses, but on which we cannot really confirm many in a practical application.

2) The author talked about training intensity and several thousand pounds lifted per week. I see the point of using trained individuals to expand on knowledge of HMB, however, I don’t see the relevance of the actual number of pounds lifted to the argument. Training stimulus is individual; if a person achieves over-reaching (and of course, there’s debate on this definition as well), then they’ve achieved an over-reached state, and the poundage is irrelevant. To imply that your target population requires the mental/physical fortitude to achieve a certain load to reproduce your results is the same as a magician saying, “Well, for this trick to work, I have to make you believe you’re a chicken,” but when challenged, later adds, “But only in my theater.” If training at loads that would otherwise induce over-reaching through undulation is a required state to these results to be achieved, then it technically shouldn’t matter how much weight is being lifted, so long as the person can “over-reach”, and/or undulate.

Even by percentage, the increases in strength observed in this study would be considered rare, even in gifted individuals. If the conditions under which HMB-FA works are so specific so as to be unreproducible in the wild, then we simply have a trial of efficacy here, as opposed to one of effectiveness, and thus, this is just promising data, on which, not a lot can be actionable. Which is fine. We all have to start somewhere. But that doesn’t mean you should go and buy this stuff when it becomes available. Unless you’re a ‘gifted responder’.