Why “no evidence of benefit” is not the same as “no benefit” in recovery science
Nov 1, 2026 · @Don
The question I keep getting
A new meta-analysis on recovery modalities is circulating on social media with a simple message: ice baths, compression, massage and the rest don’t work. That’s not what the paper says. And even if the paper were perfect, a group average can’t tell me what a recovery tool will do for the athlete standing in front of me.
The study is Jin and colleagues, published in Frontiers in Sports and Active Living in August 2026. It pooled randomized trials of physical recovery modalities in competitive and elite endurance athletes. The headline finding is that no modality was shown to outperform passive rest.
I’ve spent 40 years in athletic training rooms, labs and locker rooms. I want to explain why “not shown to outperform” and “doesn’t work” are very different statements. My argument has three parts: the trials can’t be powered, the athletes won’t accept placebo arms, and the average hides the individual. Then I’ll describe what a coach should do with that uncertainty.
What the paper actually says
The authors found limited evidence, not evidence of no value. They say so themselves, and they are unusually candid about how thin their data are.
The review included 16 trials with 273 athletes in total. Most trials measured only two or three outcomes, so each individual analysis drew on far fewer studies. Of 55 treatment comparisons across the analyses, 35 rested on a single small trial. Of 47 comparisons against passive rest, 31 were rated very low certainty and 15 low. When the authors restricted the data to confirmed elite athletes, half of the outcomes could no longer be analyzed at all.
The soreness result shows how fragile this is. With all five soreness trials included, nothing beat rest. Remove two trials whose values were estimated rather than extracted, and four modalities suddenly beat rest by nearly three standard deviations. The authors trust neither version, and neither should we.
There is one place where the null is real. For next-session performance, compression garments, active recovery and electrical stimulation showed no meaningful benefit, and compression was the only finding rated moderate certainty. I take that seriously. If an athlete expects compression tights to make tomorrow’s race faster, the evidence says probably not.
Most importantly, the authors wrote that this does not mean the modalities are useless. It means efficacy on these outcomes can’t currently be the justification. They named athlete preference, perceived benefit and cost as legitimate grounds for a decision. That sentence never made it into the social media posts.
The other side of the ledger: evidence of benefit
The Jin review is not the only evidence, and it shouldn’t be read as if it were. Earlier meta-analyses, drawing on much larger pools of studies, have found benefits for most of these tools.
Dupuy and colleagues pooled 99 studies and found massage was the most effective technique for reducing soreness and perceived fatigue. Active recovery, compression, water immersion and cryotherapy also reduced soreness, by small to large amounts. Machado and colleagues found cold-water immersion modestly better than passive rest for soreness, with the best results at 10 to 15°C for 10 to 15 minutes. A dose-response pattern like that is harder to explain as expectation alone.
The Jin authors don’t dismiss this work. They say their findings challenge the confidence placed in earlier trials more than the trials themselves. Part of the gap comes from who was studied. Earlier reviews include recreational athletes and non-athletes, who have more room to recover and more room for a tool to help.
There is also real-world evidence. Professional teams and Olympic programs spend heavily on these tools, and companies invest significant money developing and marketing them. That investment is proof of evidence. It reflects something real: athletes report feeling better, keep using the tools, and keep paying for them. We can’t ignore that signal. As a scientist, it’s my job to evaluate all of the evidence and make a qualified recommendation to my clients.
What that evidence can’t yet tell us is how much of the benefit comes from the tool’s physiology and how much from belief, or which athletes benefit most. That is a reason to keep studying these tools, not to dismiss them. This is where the Bayesian approach earns its keep. The broader body of evidence becomes the prior, and the elite data update it, so neither one gets thrown away.
Problem one: you can’t build a big enough study
Elite athletes are, by definition, rare. A national team might have a dozen cyclists or 30 skiers. The average trial in this review had about 17 participants, and that is close to the ceiling, not a failure of effort.
Small samples matter because the effects we care about at the top of sport are small. A 1 percent improvement can decide a podium, but detecting a 1 percent change against normal day-to-day variation takes far more athletes than any program can supply. A small trial that finds nothing hasn’t shown the tool fails. It has shown the study couldn’t see an effect that size.
Elite athletes also add a second problem. They may recover faster on their own, which leaves a narrower window for any tool to help. The authors raise this point themselves. So the people we most want to study are the hardest to study, and the effect we are looking for is the hardest to detect.
Coaches and programs also won’t hand over a competitive season to a research protocol. Recovery trials happen around training camps and lab sessions, not during the moments that matter most. What gets measured is a convenient slice of the athlete’s life, not the congested competition block where recovery actually decides outcomes.
Problem two: athletes won’t sign up to be the placebo
The gold-standard answer to the expectancy question is a sham-controlled trial. In practice, competitive athletes resist that design hard. You are asking someone whose livelihood depends on recovering well to spend part of a training block on a treatment designed to do nothing. Most decline, and the ones who agree are not a random sample.
Blinding is the other wall. An athlete knows whether they sat in 10°C water or 35°C water, and whether they were massaged. The outcomes these trials measure most often, soreness and perceived exertion, are exactly the ones most sensitive to belief. The review’s authors make this point too.
When researchers have managed a sham, the results are worth facing honestly. Broatch and colleagues gave recreationally trained men either cold water, plain warm water, or warm water with a “recovery” additive that was actually just soap. The fake additive matched cold water for strength recovery over 48 hours and beat plain warm water. A sham in the Jin review also ranked at or above most active modalities.
I don’t read that as “it’s all in their heads, so it doesn’t matter.” I read it as evidence that belief and expectation drive real, measurable recovery. The athlete who believes in a recovery ritual and recovers better has recovered better. What the sham data change is why the tool helps, and how much we should pay for it. They don’t change whether the athlete benefited.
Problem three: the average hides the person in front of you
A randomized trial answers one question: what happens on average. Coaching asks a different one: what happens for this athlete. Those are not the same question, and a group null doesn’t answer the second.
Picture a tool that helps a third of athletes a lot, does nothing for a third, and slightly hurts the rest. Average them and you get roughly zero. Report that as “no effect” and you have thrown away the athletes who benefit most. The person in front of me might sit far to the right of the mean, or far to the left. The group average can’t tell me which.
I need to be careful here, because this argument is easy to overstate. Much of what looks like “responders” and “non-responders” is just noise. Measurements bounce from day to day, and a single before-and-after comparison will always scatter people around the mean. Atkinson and Batterham showed that true individual differences exist only when the spread of change in the treatment group is clearly larger than the spread in the control group. When researchers apply that test to training studies, the true individual variation often shrinks or disappears.
So the honest version of my claim is not “some athletes are definitely hyper-responders.” It is this: group trials of 17 people cannot rule individual response in or out. Hecksteden and colleagues showed that repeated testing within each person can reveal individual response that a single measurement would hide. And Robinson and colleagues argue the best design for this question is the n-of-1 trial, where each person tries each option more than once. That is a method coaches can actually use.
The fix: better statistics for small populations
If elite trials can’t get bigger, the analysis has to get smarter. The standard approach asks one yes-or-no question: is the result “significant” at p < 0.05? With 17 athletes, a real but small benefit will usually fail that test. Bayesian statistics ask a more useful question: given everything we know, how likely is it that this tool helps, and by how much?
Bayesian methods do three things that matter here. First, they start from a “prior,” a formal statement of what earlier research already suggests. Results from larger trials in sub-elite athletes don’t get thrown away. They become the starting point, and the small elite trial updates them. Second, they report the probability that an effect is large enough to matter, rather than a pass-or-fail verdict. A coach can work with “there’s a 70 percent chance this improves sprint recovery” in a way they can’t work with “not significant.” Third, the same approach scales down to a single athlete, which is the question coaches actually have to answer.
This has already been done for recovery. Hecksteden and colleagues at Saarland University tested cold-water immersion in athletes using a replicate crossover design, in which each athlete did the treatment and the control more than once. They analyzed it with a Bayesian model that used earlier research as the prior. Cold water improved recovery of the first 5 meters of a sprint, though not the full 30 meters. Because each athlete was tested repeatedly, the team could also give every athlete feedback on their own response.
That last point addresses the participation problem too. Athletes resist being someone’s placebo arm. They are far more willing to join a study that tells them personally whether a tool works for them. The design turns research from a sacrifice into a service.
The Bayesian approach has a track record outside sport as well. Power posing became a symbol of failed psychology research after replications came up empty. Then Gronau and colleagues ran a Bayesian meta-analysis of six preregistered studies. They found very strong evidence that power posing changes how powerful people feel. The evidence was only moderate among participants who didn’t already know about the effect, which is the same expectancy question this recovery literature faces.
Bayesian methods are not a magic wand. A prior is a choice, and a poorly chosen or overly optimistic one can produce whatever answer you want. Priors drawn from recreational athletes may not apply to elite ones. The safeguard is to state and justify the prior before the data come in, and to show how the conclusion changes under a skeptical prior. Done that way, Bayesian analysis doesn’t lower the bar for evidence. It asks the right question for the population we actually work with.
What I do instead: treat each athlete as a study of one
If the literature can’t answer the individual question, the athlete has to help answer it. I treat a recovery tool as a hypothesis to test in that athlete, not a product to prescribe.
The process is simple:
- Establish a baseline first. Track morning HRV, resting heart rate, sleep, soreness and a readiness rating for a few weeks before changing anything, so you know what normal noise looks like.
- Introduce one tool at a time, after comparable sessions. Changing three things at once tells you nothing.
- Alternate. Use the tool after some hard sessions and skip it after others, ideally more than once each. That is the n-of-1 design in a training room.
- Judge the change against that athlete’s own day-to-day variation, not against a population chart. A shift smaller than their normal noise is not a response.
- Look at what the athlete performs, not only what they feel. Session quality and next-day output matter more than a soreness score.
Then weigh the result against cost and risk. A cheap, low-risk tool the athlete believes in and responds to is easy to keep, even if part of the benefit is expectation. An expensive tool with no measurable signal after a fair trial is a good place to save time and money. Some tools also carry trade-offs that deserve a conversation, such as timing cold exposure around sessions where adaptation is the goal.
This approach respects the evidence. It accepts the paper’s one solid finding about compression and performance. It also refuses to turn “not proven on average” into “worthless for everyone.”
The bottom line
Research tells us where to start. Measurement tells us where to go with each athlete.
The Jin review is useful precisely because it is honest about its limits. It shows how little we can claim from the randomized evidence, in either direction. People who share it as proof that recovery tools are a scam are making the same mistake as the companies that market those tools as proven. Both turn uncertainty into certainty.
My job is not to defend every gadget, and it is not to dismiss them either. It is to help the athlete in front of me find out what actually works for them, measure it honestly, and stop paying for what doesn’t.
Sources
- Jin Y, Pan D, Li M, Zhu D, Wen J. Physical recovery modalities in competitive and elite endurance athletes: a systematic review and network meta-analysis. Front Sports Act Living. 2026;8:1876924.
- Broatch JR, Petersen A, Bishop DJ. Postexercise cold water immersion benefits are not greater than the placebo effect. Med Sci Sports Exerc. 2014;46(11):2139–2147.
- Atkinson G, Williamson P, Batterham AM. Exercise training response heterogeneity: statistical insights. Diabetologia. 2018;61:496–497. (Builds on Atkinson and Batterham, Exp Physiol. 2015;100:577–588.)
- Hecksteden A, Pitsch W, Rosenberger F, Meyer T. Repeated testing for the assessment of individual response to exercise training. J Appl Physiol. 2018;124(6):1567–1579.
- Robinson ZP, et al. N of 1: optimizing methodology for the detection of individual response variation in resistance training. Sports Med. 2024;54(8):1979–1990.
- Hecksteden A, Forster S, Egger F, Buder F, Kellner R, Meyer T. Dwarfs on the shoulders of giants: Bayesian analysis with informative priors in elite sports research and decision making. Front Sports Act Living. 2022;4:793603.
- Gronau QF, van Erp S, Heck DW, Cesario J, Jonas KJ, Wagenmakers EJ. A Bayesian model-averaged meta-analysis of the power pose effect with informed and default priors: the case of felt power. Compr Results Soc Psychol. 2017;2(1):123–138.
- Dupuy O, Douzi W, Theurot D, Bosquet L, Dugué B. An evidence-based approach for choosing post-exercise recovery techniques to reduce markers of muscle damage, soreness, fatigue, and inflammation: a systematic review with meta-analysis. Front Physiol. 2018;9:403.
- Machado AF, Ferreira PH, Micheletti JK, et al. Can water temperature and immersion time influence the effect of cold water immersion on muscle soreness? A systematic review and meta-analysis. Sports Med. 2016;46(4):503–514.