http://www.news-medical.net/news/20120831/Study-reports-flaws-in-design-analysis-and-interpretation-of-Lyme-disease.aspx
Study reports flaws in design, analysis, and interpretation of Lyme disease
Published on August 31, 2012 at 9:50 AM · 1 CommentStatistical analysis questions evidence discouraging retreatment, coverage
Most doctors treat Lyme disease with antibiotics for two to four weeks after diagnosis, but if symptoms persist after that, medical guidelines recommend against antibiotic retreatment. That recommendation may not be warranted. A newly published statistical review of the four studies upon which those guidelines are based reports flaws in design, analysis, and interpretation that call into question the strength of the evidence against retreatment.Allison DeLong, a biostatistician at Brown University's Center for Statistical Sciences and lead author of the study published online Aug. 19, 2012, in Contemporary Clinical Trials, said the four studies do not prove that retreatment does not work. That questionable interpretation, however, has led doctors to forgo treatment and insurance companies to withhold reimbursement.
"The goal of the paper is to clarify what was actually found from these clinical trials and what could be said and what couldn't be said," DeLong said. "A lack of evidence should not be used to deny treatment when the studies have serious flaws."
Evidence in the trials is most often inconclusive, she and three co-authors found. Two studies even found some statistically significant benefits from antibiotics.
DeLong has been curious about Lyme disease retreatment for more than a decade since a friend of hers seemed to benefit from therapy. Her friend paid for the treatment out-of-pocket. Statisticians would call that anecdote an "n of 1," but the example stuck with DeLong as more people, including journalists, began to question whether retreatment really was ineffective.
In 2009 and 2010, DeLong and her colleagues decided to look into the matter with full statistical rigor. Their analysis started by scanning the medical literature for any randomized clinical trials offering evidence of the efficacy of antibiotic retreatment for Lyme disease. Careful review of more than 100 studies ultimately whittled the field down to the four studies on which the Infectious Diseases Society of America and the American Academy of Neurology are based their guidelines.
The most influential studies were conducted by Klempner et al., and published together in the New England Journal of Medicine in 2001. The multicenter trials enrolled chronic Lyme sufferers with positive or negative blood serum results for Immunoglobulin G, an antibody that might indicate active infection. In each of the IgG positive and negative groups, patients either received IV antibiotics followed by oral antibiotics or IV placebo followed by oral placebo. Their symptoms were measured along the way using a subjective set of health quality-of-life measures called the SF-36.
Although Klempner et al. found no significant benefit to retreatment, findings from subsequent SF-36 studies in chronic illnesses provide evidence that the Klempner study was looking for unrealistically large differences.
"The trials, as designed, called for treatment effects considerably larger than the minimum clinically important differences (MCID) identified in other chronic illnesses, suggesting that the sample sizes were inadequate and the trials were very likely underpowered to detect the true underlying MCIDs," DeLong and her co-authors wrote in the journal.
Klempner's statistics showed that treatment might or might not have been effective given the broad range of a statistical measure known as the confidence interval, DeLong said.
In another of the four trials conducted by Krupp et al.,
researchers found that retreatment produced a significant benefit for
fatigue, but the authors of the study mistakenly discounted that result,
DeLong said.
The authors became concerned that their results were tainted by too many subjects realizing that they were receiving real treatment instead of the placebo. The measure of fatigue is subjective and could be influenced by that realization. But DeLong found that the subjects weren't likely to have realized anything. Here's why: If the members of each group have a blindly optimistic seven in 10 chance of believing that they received real medicine, then the people who really were would be right seven out of 10 times and the people receiving the placebo would only be right 3 out of 10 times. The people receiving the medicine would seem like they had discovered their status, but in reality they were only making a lucky, optimistic guess.
While the Krupp study was adequately powered to measure a significant benefit from fatigue, it had less power to measure the two other treatment effects it considered: improvements in cognitive processing and clearance of a potential Lyme disease biomarker, DeLong said.
The last of the four studies, by Fallon et al., had a very small sample size. It found hints of some benefits from retreatment but nothing definitive either positively or negatively.
Ultimately, DeLong said, the best evidence to support or refute antibiotic retreatment will come when scientists develop a definitive test for active Lyme disease infection. In the interim, it is possible that chronic Lyme disease patients harbor an ongoing infection that antibiotics could treat.
"The interpretation of the trials goes too far," she said. "You can't say it's been shown that retreatment is not beneficial. You can't then jump to the conclusion that this shows there is no persistence of infection."
Source: Brown University
Phyllis Mervine
says:
The authors became concerned that their results were tainted by too many subjects realizing that they were receiving real treatment instead of the placebo. The measure of fatigue is subjective and could be influenced by that realization. But DeLong found that the subjects weren't likely to have realized anything. Here's why: If the members of each group have a blindly optimistic seven in 10 chance of believing that they received real medicine, then the people who really were would be right seven out of 10 times and the people receiving the placebo would only be right 3 out of 10 times. The people receiving the medicine would seem like they had discovered their status, but in reality they were only making a lucky, optimistic guess.
While the Krupp study was adequately powered to measure a significant benefit from fatigue, it had less power to measure the two other treatment effects it considered: improvements in cognitive processing and clearance of a potential Lyme disease biomarker, DeLong said.
The last of the four studies, by Fallon et al., had a very small sample size. It found hints of some benefits from retreatment but nothing definitive either positively or negatively.
Ultimately, DeLong said, the best evidence to support or refute antibiotic retreatment will come when scientists develop a definitive test for active Lyme disease infection. In the interim, it is possible that chronic Lyme disease patients harbor an ongoing infection that antibiotics could treat.
"The interpretation of the trials goes too far," she said. "You can't say it's been shown that retreatment is not beneficial. You can't then jump to the conclusion that this shows there is no persistence of infection."
Source: Brown University
Phyllis Mervine
says:
Congratulations to Allison DeLong and
her colleagues for showing that the emperor has no clothes. Doctors have
long used the IDSA interpretation of the NIH trials as an excuse for
medical neglect. Now, hopefully, no more.
When I was on the NIH Advisory Panel for Klempner’s clinical trial, about a decade ago, the other patient representative and I expressed concern that the results, if negative, would be used as an excuse to deny patients care. Phil Baker, NIH Lyme Program Officer at the time, assured us that the trial could do no more than determine that this particular treatment worked, or didn’t work, in this particular subset of patients. Perhaps naively, we believed him. The entire Advisory Panel knew the study design didn’t permit a broad interpretation.
NIH conveniently forgot their promise after the Data Safety and Monitoring Board announced its findings. Within days, NIH published news releases and clinical alerts with this headline: “Chronic Lyme Disease Symptoms Not Helped by Intensive Antibiotic Treatment.” I suggested they substitute "Chronic Lyme Disease Study Shows 3 Months of Antibiotic Treatment Inadequate." No one responded to my letter.
A few months later the NEJM published the paper in which the authors also leap to the conclusion, “It is unlikely that more prolonged antibiotic therapy or a different combination of antibiotics would result in greater improvement than was observed in this study.” As DeLong points out, there was no evidence for this wild conjecture.
Interestingly, the news release also stated that the investigators found that “the impact of Lyme disease on physical health was at least equal to the disability of patients with congestive heart failure and osteoarthritis.” This contradicts the IDSA claim that chronic Lyme is no more serious than the aches and pains of daily living.
The Klempner study was never designed to settle the question of the efficacy of long-term treatment, yet more than 10 years later it is still being used to deny care to seriously ill people. DeLong proves that the patient advocates were right all along. It’s time for NIH to admit it.
When I was on the NIH Advisory Panel for Klempner’s clinical trial, about a decade ago, the other patient representative and I expressed concern that the results, if negative, would be used as an excuse to deny patients care. Phil Baker, NIH Lyme Program Officer at the time, assured us that the trial could do no more than determine that this particular treatment worked, or didn’t work, in this particular subset of patients. Perhaps naively, we believed him. The entire Advisory Panel knew the study design didn’t permit a broad interpretation.
NIH conveniently forgot their promise after the Data Safety and Monitoring Board announced its findings. Within days, NIH published news releases and clinical alerts with this headline: “Chronic Lyme Disease Symptoms Not Helped by Intensive Antibiotic Treatment.” I suggested they substitute "Chronic Lyme Disease Study Shows 3 Months of Antibiotic Treatment Inadequate." No one responded to my letter.
A few months later the NEJM published the paper in which the authors also leap to the conclusion, “It is unlikely that more prolonged antibiotic therapy or a different combination of antibiotics would result in greater improvement than was observed in this study.” As DeLong points out, there was no evidence for this wild conjecture.
Interestingly, the news release also stated that the investigators found that “the impact of Lyme disease on physical health was at least equal to the disability of patients with congestive heart failure and osteoarthritis.” This contradicts the IDSA claim that chronic Lyme is no more serious than the aches and pains of daily living.
The Klempner study was never designed to settle the question of the efficacy of long-term treatment, yet more than 10 years later it is still being used to deny care to seriously ill people. DeLong proves that the patient advocates were right all along. It’s time for NIH to admit it.
