Are you a woman? Do you have breasts? Have you had breast cancer? Do you know a woman who has breast cancer? Worry about breast cancer? Don’t. Take this pill.
Researchers ran a randomized controlled trial with women who’d been treated for early stage breast cancer, assigning them to either a placebo pill or a new pill, letrozole. The new pill is designed to prevent recurrence of breast cancer. This went on for 10 years. That’s a long time, but you need to remember:
While many women with early-stage breast cancer live for a long time, they “face an indefinite risk of relapse,” said the study’s lead author, Paul Goss, director of breast cancer research at Massachusetts General Hospital.
An “indefinite risk of relapse.” That from a scientist named Paul Goss at Mass Gen and a professor of medicine at, can you be surprised, Harvard. An “indefinite risk of relapse” is the best guess this expert can make about recurring breast cancer. He says you have a risk of recurrence, but that risk is “indefinite.” Is he selling himself, a pill, or snake oil? An “indefinite risk of relapse.” Apparently he’s forgotten about staging cancers and the 5 year survival curves and all those numbers that are for him, indefinite.
Let’s believe him, be good girls, and take either the placebo pill or the real pill. Count the recurrences.
I can’t do that right now. I’m getting all this information from an article in the Washington Post from a reporter who is permitted early access to research papers that researchers like me are not allowed to read now. The paper will be published in the NEJM tomorrow (as I write this on Sunday, June 5, 2016). That means this information is being released to the public before anyone in the field with any expertise has read it. I’m going with the information in the WaPo article right now. I’ll read the paper as soon as NEJM releases it. [Publisher’s Note: I’ve now read the paper. The reporter got it right on the main points. I’ll add details from my reading of the research paper in bracket sidebars. Otherwise, everything else is my original analysis based on the WaPo reporting. End of Publisher’s Note.]
Back to the press and the counting!
The study enrolled 1,918 postmenopausal women in the randomized Phase 3 trial. All had taken letrozole for about 5 years; some also had been treated previously with tamoxifen. Half of the women were given a daily pill of letrozole for an additional 5 years, while the rest were put on a placebo. At a six-year followup, a total of 165 women had experienced a recurrence of breast cancer or developed a new cancer in the opposite breast. Of those, 67 were in the letrozole group while 98 were in the placebo group.
That meant the risk of disease recurrence and new cancer was 34 percent lower among women who continued the aromatase inhibitor for 10 years compared with the other group, researchers said.
Let’s count this more honestly. Assume that 959 women got the placebo and 959 got the real pill. In the placebo group, 98 of 959 (10%) had recurrence. In the treatment group, 67 of 959 (7%) had recurrence. That’s a 3 point difference. The Windowpane here counts a 10 point difference as Small, so this is one third of a Small effect, about a 48/52 difference. Keep counting.
However, women who were treated with the drug for a total 10 years didn’t live longer than those who were given a placebo in the study.
Don’t worry about that. It’s not true in the future.
Goss said at a news briefing on Sunday that he’s confident a survival benefit will emerge in the data in coming years.
Recall that Dr. Goss cannot count the risk of cancer recurrence as anything other than “indefinite” but he can count future survival. We call that persuasive math. No numbers, but verbal quantities. Keep counting.
Letrozole has side effects, and the women who took it for a prolonged period as part of the study suffered more bone pain, fractures and the onset of osteoporosis compared with the women who didn’t get the drug.
So, the new pill is not benign. You won’t live longer if you take it, but you may break your leg just getting out of bed one morning.
[Sidebar from Reading the Research Article and Not the News.]
Dr. Goss is extremely persuasive in his counting and reporting. He notes the bone side effects with some numbers, but reports no statistical analysis. I did. Let me quote this guy who thinks he knows persuasion.
Among 133 patients receiving letrozole who had a fracture during the trial period, 56% were taking bisphosphonates, 90% were taking a calcium supplement, and 86% were taking a vitamin D supplement; among 88 patients receiving placebo who had a fracture during protocol therapy, 55% were taking bisphosphonates, 86% a calcium supplement, and 88% a vitamin D supplement.
Can you immediately tell the main point from this presentation? Of course not. Goss buries the truth with a bunch of numbers strung together without statistical comparison.
Count 133 women on letrozole with bone fractures. Count 88 women on the placebo with bone fractures. Now, run a z-test on the difference between proportions (13.9% versus 9.2%). You get z = 4.57, p < .000001, and a h effect of .15 where .2 is Small.
Goss does honestly tell you that the pill caused more bone fractures, but he doesn’t count that difference. The effect of the pill on bone fractures is larger (!!!) than its effect on cancer recurrence (!!!).
But, wait, there’s more with Dr. Goss. He also counted women who quit the experiment. He notes.
Few women discontinued treatment because of toxic effects (5.4% in the letrozole group vs. 3.7% in the placebo group).
At least you see the numbers, but again you don’t see a statistical analysis. Guess what? Run a z-test on the difference between proportions and women in the letrozole group were more likely to quit than the placebo women; z = 2.53, p < .01, h = .08. That h effect is half of Small, but still larger than the effect of letrozole on cancer recurrence.
When Dr. Goss has numbers that make him look good, he reports the numbers and a test of statistical analysis although never with Windowpanes. When Dr. Goss has numbers that make him look bad, he reports just the numbers and no test of statistical analysis to demonstrate just how bad he looks.
You see the persuasion play. If his numbers have tests of statistical analysis, they are meaningful. If his numbers don’t have tests of statistical analysis, they are not meaningful. He wants you to draw the inference based on the Cue of statistical testing. Good news has it. No news doesn’t.
Technically this is not lying. Technically this is not deception. Technically this is also not the truth. That’s how you use persuasion with statistics. You change how Other Guys think about the data with your pattern of reporting. Use statistical significance to underline your good news while omitting statistical significance to hide the bad news.
[End of Sidebar from Reading the Research Article and Not the News.]
Stop counting now and start thinking. A new drug aimed at preventing that “indefinite risk of recurrence” produced a 3 point advantage over a sugar pill, increased more side effects with bone pain, fractures, and osteoporosis, caused more women to quit the study, and made no mortality difference over 10 years. You know what that means?
The results of the study were published online in the New England Journal of Medicine, which also featured an editorial calling the study “reassuring” and saying that “the findings have direct application for clinical practice.”
Whew! That’s a relief. We’ve got a new pill girls can take that counting shows makes no difference in life and death, a trivial difference in recurrence, with the added benefit of more bone fractures. And, I thought there was a problem. I’m clearly wrong. Remember.
Goss said at a news briefing on Sunday that he’s confident a survival benefit will emerge in the data in coming years.
. . . an editorial calling the study “reassuring” and saying that “the findings have direct application for clinical practice.”
But this time the wolf, the werewolf, is your doctor writing in the NEJM. The count clearly does not add up. No difference in mortality. Extremely small difference in recurrence. Bad bone side effects. The only good news is from the words, the persuasive words from Dr. Goss and editorialists at NEJM. When the data don’t fit, you must admit: Persuasion!
I’ve lived through enough bad data to understand the psychology behind Goss and others cheering this on. The dissonance is overwhelming. All that work and for this piddling pool of piddle. Then you start thinking about it and suddenly you realize the results are better than they seem and you know, you just know in your bones, that they will be even better in the future. You could call that wishful thinking, but it’s dissonance reduction. Goss can’t handle the truth of the inconsistency between his hypothesis (and all the funding he got) and his results.
Of course, when you think you can publicly declare “an indefinite risk of recurrence” and that no one is laughing, then you are deep in the rabbit hole. Dr. Goss looks both incompetent and untrustworthy while thinking he’s getting away with it.
Look at this panthering. The guy takes this lousy data to a national conference, presents a slide show, then does a press conference. He gets a favorable editorial from NEJM to support his rosy persuasion. And, he gets a head start on everyone else because no one can read the paper for 24 hours. Except for news reporters. Look at the headline he got at the Washington Post.
I commend the writer of this article who provided a much more balanced take on Goss’s persuasion than the headline writer. But, what do you think people are going to remember? That headline or the reporter’s thoughtful writing?
This is how you advance careers and programs and institutions. Of course, it also kills or maims Other Guys, but just get insured and let the lawyers handle that.
Paul E. Goss, M.D., Ph.D., James N. Ingle, M.D., Kathleen I. Pritchard, M.D., Nicholas J. Robert, M.D., Hyman Muss, M.D., Julie Gralow, M.D., Karen Gelmon, M.D., Tim Whelan, B.M., B.Ch., Kathrin Strasser-Weippl, M.D., Sheldon Rubin, M.D., Keren Sturtz, M.D., Antonio C. Wolff, M.D., Eric Winer, M.D., Clifford Hudis, M.D., Alison Stopeck, M.D., J. Thaddeus Beck, M.D., Judith S. Kaur, M.D., Kate Whelan, M.Sc., Dongsheng Tu, Ph.D., and Wendy R. Parulekar, M.D. (2016). Extending Aromatase-Inhibitor Adjuvant Therapy to 10 Years. Published online, June 5, 2016, New England Journal of Medicine.
Rowan T. Chlebowski, M.D., Ph.D., and Matthew J. Budoff, M.D. (2016). Changing Adjuvant Breast-Cancer Therapy with a Signal for Prevention. Published online, June 5, 2016, New England Journal of Medicine.
P.S. Rowan T. Chlebowski and Matthew J. Budoff wrote the favorable NEJM editorial. As part of publishing in many journals nowadays authors complete a conflict of interest form that asks about research, financial, or other factors that might affect or appear to affect the content of the information. Chlebowski and Budoff noted (pdf) a financial interest – Chlebowski has an NIH grant, while Budoff is taking consultant money from many Big Pharma guys. But nothing else. No other relationships.
Except this. Do a search at PubMed on author names Goss and Chlebowski and you find 11 coauthored papers.
That’s not a relationship that might have a conflict of interest? This is like me writing an editorial for a paper that my research partner, Bill Reger, wrote. Yeah. No conflict of interest at all when co-authors comment on each other’s work.
Dr. Budoff has no publications with Dr. Goss. Of course, Dr. Budoff has no publications on cancer either. PubMed shows him with nearly 600 papers, none with the word cancer in the title.
Good grief. I’ve got more publications with the word, cancer, in the title! Just one! But one more than Budoff.
Dolin, D., & Booth-Butterfield, S. (1995). Cancer prevention and the foot-in-the-door technique. Health Communication, 7, 55-66.
Perhaps I’d be qualified to understand this research if my cancer research was on the Pill-in-the-Mouth technique or the Hiding Bad Statistics tactic.
P.P.S. Paul Goss asserts (pdf) that he has no conflicts of interest. Yet, he notes that the pharma, Novartis, supplied funding for this RCT. Novartis is selling the new pill, letrozole. And, just for fun, do a Google search on the key terms of Novartis and Paul E. Goss.
Scan over the results and think about the nature of the relationship between Novartis and Goss. Man, where’s Marion Nestle when you really need her?