Enslaved to evidence-based approaches? Reminds me of nothing more than William Dembski's pathetic level of detail.Medicine has become enslaved to "evidence-based" approaches that rely on randomized, clinical trials as the only measure of whether a treatment is valuable[.]
Pathetic level of slavery to evidence
Andrew Weil practices "alternative" medicine, that is, medicine for which there is no evidence of efficacy. Now, according to an article in the Los Angeles Times, he recommends that evaluations of the efficacy of a treatment include "[p]atient factors -- including how patients felt about the treatment, whether they can afford it and any evidence of a placebo response" (the words are those of the reporter, not Weil).
Some of Weil's other recommendations, such as consideration of the funding source and possible unintended consequences, make sense, but "how patients felt about the treatment" is an invitation to peddle snake oil - maybe we should perform all clinical trials using red pills to get the best outcome.
Paraphrasing Weil, the article goes on to say,
63 Comments
Joel · 16 May 2011
"Medicine has become enslaved to “evidence-based” approaches . . . "
Just like engineering, architecture, aviation, agriculture and seismology are 'enslaved' to "evidence-based" approaches. Evidence-based approaches have served these fields well--why abandon them in the case of medicine?
mrg · 16 May 2011
And, of course, the law ... "your honor, I would like to offer 'non-materialistic' evidence in favor of my client." "Denied."
Not to mention businesses, emergency response organizations, and absolutely the military: a warfighter that doesn't care about the real facts is going down in flames.
Glen Davidson · 16 May 2011
And don't get me started on how enslaved to evidence-based approaches the judiciary is.
Although, I always like to imagine IDiots and people like Weil being consistent in their anti-evidence shtick when accused of some serious crime. Do you think that any of them will clamor for the use of "revelation" or "it looks like he's guilty" when their own freedom is on the line?
Somehow, I think that they'll challenge every "non-materialistic" and "non-evidence-based" claim made against them, although they might very well try to use nonsense as a defense.
Nothing new there, though. They always demand rigorous evidence from others, while they feel exempt from the demands of scientific rigor themselves. They'll hold us to sound standards, it's just that they have "higher ways of knowing" than their opponents.
Glen Davidson
Stanton · 16 May 2011
mrg · 16 May 2011
I think for any practical endeavor any complaints about "materialism" are silly: "Ya'll lost that battle a long time ago, people."
On that same coin, I do get a little weary of overblown claims for "Science" with a capital "S" ...
is it really any more or less "evidence-based" than any other practical exercise? And this commitment to
evidence is what exercises the critics. Again: "You've lost. You can complain forever but you can't
win. Get some sense and give it up."
Science Avenger · 16 May 2011
The people critical of "evidence-based approaches" can never answer the big question:
If you were wrong, how would you know it?
They simply assume that their internal truth detector is infallible. Gee, now where on earth do you suppose they were first introduced to that concept? So much for the "what's the harm" arguments.
eric · 16 May 2011
I suspect Weil would have the same problem defending his position in court that Behe had, when Behe proposed a broader definition of science. The opposing lawyer would get up, point out that sacrificing chickens on an altar could count under Weil's definition of "effective treatment," and that would be the end of it.
harold · 16 May 2011
Weil is eccentric but should not be confused with dangerous quacks. In no way does he try to replace or prevent mainstream therapy.
He openly recommends mainstream medicine for serious complaints.
Even where he recommends what he calls "integrative" or "alternative" stuff, it is often just evidence-based mainstream stuff.
His diet and exercise advice is 100% mainstream.
This is a guy who recommends eating lots of fruit and vegetables and fish oil, exercising, and managing stress, and using mainstream medicine if you get sick anyway.
He may do it in a funny robe with a big bushy beard, and he may charge tens of dollars for books that peddle glorified common sense and other arguably useless but fairly harmless products, and he may have made a very silly statement here, but he really is not a harmful figure.
If anything, he weans a lot of well-meaning but confused people off of harmful woo.
Mike Elzinga · 16 May 2011
One can’t help thinking of homeopathic dilutions that go far beyond the number of molecules that would have to be removed from a water solution in order to leave its “essence” imprinted in the water.
There is only one motto that fits that “science:” Nothing works.
eric · 16 May 2011
harold · 16 May 2011
Stuart Weinstein · 16 May 2011
Stuart Weinstein · 16 May 2011
Stuart Weinstein · 16 May 2011
harold · 16 May 2011
Stuart Weinstein -
I believe you're thinking of this repugnant slimeball.
Glen Davidson · 16 May 2011
truthspeaker · 16 May 2011
I wonder how Dr. Weil knows when to put fuel in his car? Does he do it when it feels like the right time, or is he enslaved to an evidence-based approach that says looking at the fuel gauge is the only way to tell if his car needs gas?
truthspeaker · 16 May 2011
truthspeaker · 16 May 2011
...and you can't wean people off of more harmful kinds of woo by telling them that woo works.
Chris Lawson · 16 May 2011
This topic is right up my alley as I teach EBM to medical students and GP registrars.
On reading the article in LA Times, I think it is fair to say that the only objectionable statements were made by the reporter rather than quotes from Weil. For instance:
"Medicine has become enslaved to 'evidence-based' approaches that rely on randomized, clinical trials as the only measure of whether a treatment is valuable, Weil and his co-authors wrote."
The original paper (Scott Shannon, Andrew Weil, Bonnie J. Kaplan. Alternative and Complementary Therapies. April 2011, 17(2): 84-91. doi:10.1089/act.2011.17210 -- as the reporter couldn't be bothered to provide a proper reference) says nothing of the sort. It does make some (IMO very dubious) claims about over-emphasis of randomised control trials, but the paper recommends a grading scale of evidence for effectiveness in which the top 3 levels are based on RCTs.
The LA Times article also fails to recognise that the standard medical tiers of evidence already include non-RCT studies, such as case-control and cohort studies. The paper that first linked tobacco to lung cancer was a famous 1950 case-control study by Doll and Hill (they thought that tarmac or motor vehicle exhaust fumes caused lung cancer and discovered to their surprise that the only strong association was with cigarette smoking) and supported by the 1954 British Doctors Study, a cohort study. No RCTs in sight.
There are still many statements in the paper that I would dispute, but they are not on the same scale as the LA Times article suggests. In fact, the article pretty much misses the point of the paper, which calls for a different approach to weighing evidence, while the reporter has reconstructed it as a "mainstream evidence is bunkum" narrative.
Gary Hurd · 16 May 2011
The first post doctoral proposal I wrote was for a study of the ethnopharmacology of coca use in Peru. That was in 1976, and I became acquainted with Andy Weil, and some others like Tim Plowman at Harvard, who were of the opinion that the "war on drugs" was really a war on ourselves, and particularly a war on minorities. The DEA was then, as now, merely an armed branch of the political and religious far-right. DEA "enforcement" was then just moving into armed foreign operations in South America targeting "communists" whether or not they were involved with drug production.
The DEA became particularly enraged with Weil when he pointed out that the "war" was already lost when the quality illegal drugs like heroin went up, and the street price went down. Andy also observed anyone in the USA who had the least interest in taking heroin, cocaine, or any other drug could find a ready supply anywhere! As an MD, he was particularly vulnerable to DEA pressure. He would not retract his (obviously true) statements that drug use, drug abuse, and drug addiction were not the same thing, and that this is a medical question that cannot be "treated" with guns and prisons. (That was true 35 years ago, and it is true today). The DEA retaliation was to deny Weil a schedule II drug license so that he was effectively prevented from practicing medicine.
My opinion was that this led Andy to two rather profound questions; "What is medical practice beyond giving people shots and pills?" and "What can I do to earn a living?"
The answer to both questions was that he teach people that prevention is better than any medicine by writing books, giving talks and to sell this with a degree of flamboyance.
Some of the specific misunderstandings about medical practice, "evidence based" or not, that I have seen in the posts so far will wait until after my dinner.
rrich · 16 May 2011
Weil and co-authors illustrate the simple point that medical research, like science generally, is always an interested endeavor. Always conducted in social and cultural context (Laura Nader 1996 Naked Science). I have intimate experience with a close family member diagnosed with Multiple Sclerosis. The treatments for MS (Rebif and Copaxone) have proven largely ineffective in controlling both the occurrence of her lesions and the symptomology of the disease. We have pursued information on other treatments, but very little is establishment-sanctified. Big Pharma is the interested actor here, which pays for and conducts almost all the cited research on MS treatments. There are few "scientific" studies, for instance, on the effect of diet on MS, despite strong anecdotal evidence that diet can play a significant role in both lesion count and symptomology. See Terry Wahls’ work and her book Minding my Mitochondria if you are interested. Why would a pharmaceutical company conduct or fund a study on the effects of diet on MS when none of their products would be used, and in which a treatment could be found that would actually subvert their own research-based products? They wouldn’t, and no one would expect them to. However, as patients, our family has grown increasingly frustrated with seemingly narrow self-interested approaches to the MS research problem by a mainstream science that cannot provide answers to easy and basic questions. Our specialist does not dismiss the efficacy of diet as treatment for MS. But when we ask him about it he says he doesn’t know much about it. There is little accepted clinical research on diet and MS, and little of what there is makes it to the literature with which he is familiar. I agree with Lawson above that the LATimes reporter recontextualizes the article in a standard narrative format and misses the more interesting point of the article on an alternative system of weighing evidence of efficacy that includes funding, accessibility of the treatment, and other patient factors. Medicine and the scientific research upon which it is based is today too much a closed paradigmatic corporate loop in the worst Kuhnean sense. A critique of science and medicine as socially interested and invested in certain kinds of (pharmacological) results is one part of a healthy understanding of contemporary medicine and science.
harold · 16 May 2011
Gary Hurd · 16 May 2011
Well, Andy's employment problem was resolved at the U. of Arizona. The question of how do we measure patient outcome visa-vis drug trials was not so easily resolved.
Let me give a personal experience: some years ago I was prescribed a medicine by a urologist to prevent vomiting during treatment for a kidney disease. The drug had passed all kinds of trials with flying colors- except it did not prevent me from vomiting. And, when I did, it was a particularly nasty tasting blue vomit, much worse than the ordinary unmedicated stuff.
This is the difference between an “evidence-based” double-blind drug trial, and a clinical result.
Mike Elzinga · 16 May 2011
Gary Hurd · 16 May 2011
Mike Elzinga, you support you promotion of "factual," "empirical," "scientific" medicine with "first hand reports of this from a close relative?"
This is not the "factual,"empirical," or "scientific" evidence that the Holy Church of the Double-Blind accepts. These "first hand reports of this from a close relative" are exactly the anecdotal data that purely "scientific" medicine rejects, exactly like to many "scientific" doctors reject their patient's actual clinical experiences. (You cannot die, I used medicine to cure you!) Are we to practice medicine as physicians, or statistics undergrads?
Gary Hurd · 16 May 2011
Damn! I hate typos, even with the preview they slip by. Then seconds later, they are glaring, mocking me.
you(r)
to(o)
Mike Elzinga · 17 May 2011
The Founding Mothers · 17 May 2011
truthspeaker · 17 May 2011
harold · 17 May 2011
truthspeaker · 17 May 2011
Harold, it almost seems like you are saying that human bodies are enormously complex things, and that a drug that works for a large majority of people won't work for everybody. What a radical concept!
~
mrg · 17 May 2011
eric · 17 May 2011
truthspeaker · 17 May 2011
mrg, I'm "only" 40, and chicken pox was a very common childhood disease when I was a kid. Despite being exposed several times as a child, I never actually contracted it until I was 16.
hoary puccoon · 17 May 2011
mrg · 17 May 2011
TomS · 17 May 2011
mrg · 17 May 2011
I'm not following the objection to the objection. Clarify please?
mrg · 17 May 2011
BTW: By "risks of dying of measles" that means "risk of dying if you contract it" and not "risk of a nominally healthy person ultimately dying of measles".
harold · 17 May 2011
TomS and mrg -
In fact, the most correct statistic to use is usually "risk of a healthy person in a vaccinated population dying (or suffering whatever complication you want to use) from getting the vaccine, versus the risk of a healthy person in an unvaccinated population ever getting measles and dying (or getting a super-nasty complication or whatever)".
While vaccine denialists justify their behavior by claiming vaccines are dangerous, what mrg was presented with is actually the "free ride" argument.
The flawed logic is "you get your kid vaccinated and risk the side effects, I'll skip the vaccination, but there won't be anybody for my kid to get measles from, so I get a free ride - you take all the risk and I share the benefits".
Why is this logic flawed? It sounds cynical but valid.
Here's the problem with it - the viruses we vaccinate against are incredibly infectious. The risk from vaccination is very, very low.
Once you get, say, measles, the risk of severe complications is somewhat low, but massively higher than the risk from a vaccine. So you only need to have a very low probability of getting measles for the free ride strategy to have a negative expected value.
As soon as...
p(getting measles)*(bad outcome from measles) is greater than p(bad vaccine outcome)...
the "free ride" strategy isn't "free".
Let's put some numbers in there, using "death" as "bad outcome" for simplicity. For simplicity I'm going to assume that vaccinated people don't ever get measles. That's not strictly true, but this is a simplified model. Let's assume we're looking at a ten year period, also to keep it simple.
With ideal medical care, a measles patient still has a 1/1000 chance of dying! (The mortality rate is much higher where optimal medical care is not available, which is most depressing, since optimal medical care is mainly palliative and controlling opportunistic infections.)
There is not even a clear death rate assignable to completing an MMR vaccine schedule, but let's call it one in ten million, which is probably very high. Typically, studies just don't record enough deaths to get really reliable fatality rate.
As you can see, in this model, as soon as chances of contracting measles goes over 1/10,000, the "free rider" is actually losing - his chances of dying of measles are already higher than his chances were of dying of the vaccine. That is, if he's walking around healthy but has a 1/10,000 chance or better of getting measles, his strategy loses.
And measles is very, very contagious. Literally everybody used to get it. And it is endemic all over the world; it isn't close to being eradicated.
So in short, the free ride strategy is only a free ride is virtually no-one uses it. I don't know what the threshold percentage of kids skipping vaccines is, to make the "free ride" strategy negative - no-one does - but it's almost certainly being exceeded in the US.
In fact, the presence of "free riders" is actually strong motivation to be really sure to get vaccinated.
mrg · 17 May 2011
mrg · 17 May 2011
I might add that antivaxers like to play games with stats and it is nice to be able to see through them.
As Jeff Shallit put it: "Probability calculations are the last refuge of a scoundrel."
harold · 17 May 2011
Fake probability calculations, that is.
Valid probability calculations are a useful tool.
Gary Hurd · 17 May 2011
Speaking form a fully accredited geezer perspective, I grew up with plenty of kids who suffered various degrees of polio injury. I also had a few kinds of measles, and chicken pox, mumps, whopping cough, and scarlet fever, and valley fever- all before I was 12.
There was concerned commentary in the 1800s that "medical advances" were producing a weakened "national stock" by preserving the lives of "weak" children who would have died of common diseases. In Chapter 5 of the Decent of Man, Darwin wrote regarding the protection of the poor, weak, and ill, “Nor could we check our sympathy, even at the urging of hard reason, without the deterioration in the noblest part of our nature. The surgeon may harden himself whilst performing an operation, for he knows that he is acting for the good of his patient; but if we were to intentionally neglect the weak and helpless it could only be for a contingent benefit, with an overwhelming present evil.”
Shebardigan · 17 May 2011
harold · 17 May 2011
Chris Lawson · 18 May 2011
If anti-vaxxers claimed that we should not vaccinate children against diseases that no longer carry a significant burden in the community, they would be absolutely right and would be completely in line with standard medical opinion. This is, after all, why we no longer vaccinate against smallpox (which has been eradicated in the wild) and in Australia we no longer routinely vaccinate against TB (which has not been eradicated and even occurs occasionally in Australia, but is so rare here that it is not worth vaccinating every child).
The problem comes when anti-vaxxers lie about the evidence. I will not go through all the common anti-vax fallacies except to say that there is a surprising overlap with anti-evolution fallacies.
raven · 18 May 2011
Sylvilagus · 18 May 2011
Karen S. · 18 May 2011
I'm a boomer and I remember the polio vaccinations. And my mother remembers lots of crippled kids and even funerals for kids due to polio. Violinist Itzhak Perlman contracted polio as a young child and requires crutches to walk.
It would be nice if anti-vaxxers could travel back in time for a reality check.
W. H. Heydt · 18 May 2011
JASONMITCHELL · 18 May 2011
perhaps tangental to the original article - when drug/ therapy testing we tend to limit the variables (just good science) but sometimes that means that a more complicated experiment isn't done in oprder to prove the efficacy of a single compound. for example, let's imagine that a natural substance is believed to be an effective treatment of a malady - so extracts of the substance are prepared and tested in a double blind study and results are analyized - but tests of the substance as it is found in nature are rareley performed- why? (look at the entire 'suppliemnts' aisle in the grocery store)
mrg · 18 May 2011
MichaelJ · 18 May 2011
David Fickett-Wilbar · 18 May 2011
Nomad · 18 May 2011
Gary Hurd · 19 May 2011
Natural products cannot be patented. Extraction methods can be, and synthetic analogues of plant products (as an example) can be patented.
I helped on the early part of a project to identify an anticonvulsant from "china berry." Neither the plant, nor boiling the berry could be patented, so the only pharmaceutical company interest=money was to identify the active chemical so that a patentable synthetic could be made.
harold · 19 May 2011
rrich · 19 May 2011
Thanks for the reply harold; I appreciate your comments and wishes and the depth of your reply. I don’t agree with all that you say, but the posts have clearly moved on, so I will address just your conspiracy point because I think it is particularly relevant to why I posted in the first place. My problem with the mainstream research on MS is not that it amounts to a conspiracy, but that research is always interested, and that the interest is not only defined by what is best for patients. And , I’m sorry for the buzzwords, but the scientific community addressing MS, which is really just one of many similar kinds of communities, each of them more or less distinct, is very much a closed corporate loop. That is not a conspiracy claim. It is not a simplistic claim that all corporations are bad or that researchers conspire in dark laboratories to serve only the bottom line. It is an observation based on our experience.
The “easy” and “basic” questions I refer to are questions about research in the literature on the effects of diet on MS progression. Perhaps easy and basic were not very good qualifiers. I meant that we ask our specialist questions that from where we stand as patients seem incredibly simple for a highly trained MS specialist at a first-rate institution. In this case, the question is “what can you tell us about the effects of diet on MS progression.” The response we get to this question is not that diet has no impact; the specialist doesn’t conspiratorially try to steer us back to pharmaceutical treatments by pooh-poohing the impact of diet. Instead, the response we get is a blank stare and an awkward silence. The specialist cannot answer the question. And he can’t answer the question (he tells us honestly) because there is no research that he feels he can cite with authority that might answer our question. The question isn’t addressed in the literature he reads.
As an aside, you suggest that if the questions are so easy and basic that we ought to answer them ourselves. That is a bit unfair. First, we are of course in the process of doing so. Second, those of us who must face the medical bureaucracy in the treatment of illness and disease rely on the advice and leadership of our physicians. If my relative were told by her specialist, for instance, that he was familiar with Wahls’ research and claims and that while he knew of no research that supported them still found them interesting, his interest could have a huge impact on how she decides to treat her MS. But he had not even heard of such an approach, and this makes my relative less inclined to pursue that course. Knowing my relative and how she strives to follow the treatment prescribed by the specialist, I personally feel that if the specialist specifically prescribed diet change as part of her treatment that she would pursue that course with a passion.
I am a socio-cultural anthropologist by training, and approach our experiences with MS and the medical bureaucracy as a kind of auto-ethnographic experience as we wade through the system trying to locate ourselves in a prolific stream of corporate literature and agents. The corporate nature of the experience is unquestionable; the literature we are given is corporate-generated; the meetings and lunches we go to are corporate sponsored; our nursing assistant who helps with injections is paid by the company that has developed the drug being injected; even many of the patient groups are organized around corporate human relations staff and outreach personnel. Anthropologists use our own experiences, and the daily experiences of the people we study, to understand larger systemic processes and contexts within which experiences occur and that help to structure those experiences.
So, building on my experience, I would end by taking us back again to Weil’s basic point that qualitative data should have a greater role in the evaluative process of medical treatments. When I first read the “pathetic level of slavery” post and then LA Times article, Weil’s basic point resonated very strongly with my own experience: mainstream science could benefit from qualitative kinds of data that stem from people’s experiences. Qualitative studies could enhance (they shouldn’t replace double-blind studies) evaluative procedures. To say that scientific research is “interested” is from my perspective not a statement of conspiracy, but a recognition of how our for-profit health system is organized; it is a structural observation. And I say that not to argue against a for-profit system, nor to claim that profit is the most salient feature of our health-care system, nor to say that corporations or big pharma are bad. Profit is just one important feature of our system, and so long as one can get access, a for-profit system has significant advantages. I make the point instead because too often in my experience (and here I refer not just to my MS-related experience, but also in the posts here on Panda’s Thumb, including the original one on this blog) advocates of rationalist science try way too hard to make science itself a sacrosanct enterprise of objectivity and truth. As one who practices a form of scientific method, I think that is a diminished understanding of scientific knowledge. Thanks again for the dialogue.
qetzal · 19 May 2011
mrg · 28 May 2011
Rick · 6 June 2011
I think I'd be more comfortable going to a vet that to Mr Weil.