Who to Trust? Reading Medical Articles With a Grain of Salt

How much faith should we put in published, peer reviewed studies which show that a drug, vitamin, surgical technique or medical device produces a favorable result?

We certainly want to believe that a conclusion based on raw data generated in a properly designed test by objective scientists must be true and is therefore reliable.

A recent article in The New Yorker -- "The Truth Wears Off “ -- by Jonah Lehrer reminds us of the perils of putting all our faith in scientists who, like the rest of us, are fallible. Scientists, especially medical researchers, he contends, tend to select their data to confirm a preconceived thesis, especially if they want to publish their study in a prestigious journal.

Once a "truth" gets established – e.g, cardiac stents work, PSA tests save lives, Vitamin E is good for you, hormone replacement therapy for menopausal women works, second generation anti-depressants are effective, etc -- it's hard to un-establish it. Few journals put priority on publishing studies that show that a drug or device had "no effect," a phenomenon which Lehrer calls "publication bias."

The writer's point is not that our medical journals are rife with scientific fraud. In the real world, scientists struggle with making sense of their data. If there are anamolies that don't seem to follow a pattern, they might get tossed out. Like anyone else, a researcher is wired to want to disregard what he or she doesn't want to see, or can't explain.

As Lehrer observes: "The problem of selective reporting is rooted in a fundamental cognitive flaw, which is that we like proving ourselves right and hate being wrong." Quite simply, it feels good to prove a hunch. It feels even better, he notes, when the researcher has a financial interest in the outcome, or stands to advance his career.

The answer, Lehrer argues, is in properly designing studies and making sure the data are both transparent and rigorously gathered, even if they contradict the hypothesis. Moreover, before publishing, the scientist should lay out on the front end what's a sufficient level of proof. He also suggests the use of accessible databases (something near and dear to Dr. Cameron and the Pacific Meso Center).

Bringing the subject closer to home, for many decades now US doctors have been quick to tout the EPP as the best treatment for mesothelioma. Studies, mainly out of the Brigham and Women's hospital, have been published to prove the point. Several years ago, a big clinical trial, financed by Eli Lilly, showed that the chemotherapy drug Alimta was superior to doing nothing. Alimta went on to become the "front line standard of care" for pleural meso patients.

As you read up on the treatment options available, and listen to experts, it’s wise to stay on guard. Ask yourself, was the research based on selective reporting (aka, "cherry picking" the patients who did well but casting out those who didn’t)? Have the data been made available for review? Have the conclusions been validated elsewhere? Was the clinical trial randomized? Were apples compared to apples (if that’s even possible!).

Note that there has never been a clinical trial in the US in which pleural mesothelioma patients were randomly selected for either an EPP, a Pleurectomy/Decortication, or no treatment. Putting the ethical morass aside (I don’t think a patient would be eager to participate in a trial in which he was forced to do nothing or have his lung amputed), even if there was a well designed study, clearly surgical technique could not be 100% replicated, and every patient is different (genetics, age, sex, staging, pre-existing conditions, tumor cell type, will to live, etc).

In the end, we wind up making choices based on trust. Do we trust the doctor and his team? Do we trust the "science" on which he bases his opinions? Does he admit what he doesn't know? Does he follow the current fad or stubbornly cling to a one-size-fits-all strategy? Does he have passion without the in-your-face zeal ? Does he have a possible conflict of interest where, for example, he's got an irrepressible financial or career incentive to push one flavor over another?

Has your doctor tried to maintain “neutrality?” Radical surgery will transform the patient’s life irreparably. Has your doctor tried to suppress his own bias, anger or elation in recommending a treatment? Has your doctor truly explained whether an option first, does no, or at least very little, harm, balanced against the prospect of measurable benefit?

It’s not easy for a surgeon, or anyone, to “go Swedish” and consciously set aside biases. One thing is for certain, you’re not getting a fair shake if your doctor tells you he’s going to cure your mesothelioma. There is no proven cure for mesothelioma, period. At best, with an enlightened strategy, orchestrated by the an honest and caring medical team, meso patients can buy valuable time.

As WC Fields used to say: Trust your fellow man, but always cut the cards.

Roger G. Worthington, Esq.