A handful of circumcision advocates have recently begun haranguing the global health community to adopt widespread foreskin-removal as a way to fight AIDS. Their recommendations follow the publication of three [1] randomized controlled clinical trials (RCCTs) conducted in Africa between 2005 and 2007.
These studies have generated a lot of media attention. In part this is because they supposedly show that circumcision reduces HIV transmission by a whopping 60%, a figure that wins the prize for “most misleading possible statistic” as we’ll see in a minute. Yet as one editorial [2] concluded: “The proven efficacy of MC [male circumcision] and its high cost-effectiveness in the face of a persistent heterosexual HIV epidemic argues overwhelmingly for its immediate and rapid adoption.”
Well, hold your horses. The “randomized controlled clinical trials” upon which these recommendations are based (I use scare quotes deliberately) represent bad science at its most dangerous: we are talking about poorly conducted experiments with dubious results presented in an outrageously misleading fashion. These data are then harnessed to support public health recommendations on a massive scale whose implementation would almost certainly have the opposite of the claimed effect, with fatal consequences. As Gregory Boyle and George Hill explain in their exhaustive analysis of the RCCTs:
While the “gold standard” for medical trials is the randomised, double-blind, placebo-controlled trial, the African trials suffered [a number of serious problems] including problematic randomisation and selection bias, inadequate blinding, lack of placebo-control (male circumcision could not be concealed), inadequate equipoise, experimenter bias, attrition (673 drop-outs in female-to-male trials), not investigating male circumcision as a vector for HIV transmission, not investigating non-sexual HIV transmission, as well as lead-time bias, supportive bias (circumcised men received additional counselling sessions), participant expectation bias, and time-out discrepancy (restraint from sexual activity only by circumcised men).
That’s a whole laundry list of issues, so let me highlight a few of the more egregious. First, consider the “lack of placebo control.” What does that mean? Normally, when you’re trying to determine whether some medical intervention has a disease-fighting effect specific to its own (hypothesized) mechanisms—and over and above the placebo baseline—you have to have a control group. That group gets a dummy intervention, and nobody is supposed to know which participants were exposed to the actual treatment until after the results are in.
After all, if someone knows (or thinks) that they’re getting a great big helping of medicine, they might act in various ways—whether consciously or unconsciously—that have the effect of generating positive health outcomes but which have nothing to do with the intervention itself. In the case of circumcision, however, there’s no way not to know if you’ve received the “medicine”—you have to go through a whole surgery and then you don’t have a foreskin anymore—so this basic condition of a true clinical trial is violated in the first instance.
But that’s just the tip of the iceberg. As Boyle and Hill point out, the men who were circumcised got additional counseling about safe sex practices compared to the control group, and then they had to refrain from having sex altogether for the simple reason that their lacerated penises had to be wrapped in bandages until their wounds healed – leading to what Boyle and Hill refer to as “time-out discrepancy” in the quote above. By contrast, the non-circumcised men got to keep having sex during the full two month period during which the treatment group was in recovery mode.[emphasis added] Then, mystery of mysteries, the trials were stopped early. These issues pose serious problems for the scientific credibility of the studies. Taken together with the other flaws, here is why:
Let’s assume for a second that the circumcised men really did end up getting infected with HIV at a lower rate than the control-group men who were left intact—even though, as we will see in a moment, we have very little reason to believe that this is so. Why might that outcome have happened?
If you answered, “Because those men knew they were in the treatment group in the first place, had less sex over the duration of the study (because they had bandaged, wounded penises for much of it), and had safer sex when they had it (because they received free condoms and special counseling from the doctors), thereby reducing their overall exposure to HIV compared to the control group by a wide margin” then you are on the right track.
Step 2. How not to report results
Now why should we doubt that the circumcised men actually did have a lower rate of HIV infections in the first place, poor experiment design notwithstanding, as I suggested in the paragraph above? After all, the 60% figure that’s being thrown around in media reports is a pretty big number, and it can’t be off by that much, even if the studies had some flaws, right? Not so fast. Do you know what the “60%” statistic is actually referring to? Boyle and Hill explain:
What does the frequently cited “60% relative reduction” in HIV infections actually mean? Across all three female-to-male trials, of the 5,411 men subjected to male circumcision, 64 (1.18%) became HIV-positive. Among the 5,497 controls, 137 (2.49%) became HIV-positive, so the absolute decrease in HIV infection was only 1.31%.
That’s right: 60% is the relative reduction in infection rates, comparing two vanishingly small percentages: a clever bit of arithmetic that generates a big-seeming number, yet one which wildly misrepresents the results of the study. The absolute decrease in HIV infection between the treatment and control groups in these experiments was a mere 1.31%, which can hardly be considered clinically significant, especially given the numerous confounds that the studies failed to rule out.
Step 3. How not to make public health recommendations
So far we have been discussing problems with the experiments themselves—what’s called “internal validity” in technical terms. I really want you to read the Boyle and Hill paper here, because they go into painstaking detail about each of a long parade of flaws I can’t hope to cover in one blog post. I mean, there are a lot of flaws. Please read the paper. But let’s switch gears now and talk about the flip-side of things, or what’s called “external validity” – that is, problems with taking what you’ve (supposedly) found in a (relatively) controlled setting like an experiment and applying it to the chaotic mess that is the real world.
Lawrence Green and his colleagues published an important article on just this topic as it relates to “the circumcision solution” in the American Journal of Preventative Medicine. “Effectiveness in real-world settings,” they sensibly point out, “rarely achieves the efficacy levels found in controlled trials, making predictions of subsequent cost-effectiveness and population-health benefits less reliable.”
Some major issues with trying to roll-out circumcision in particular include the fact that the RCCT participants—who were not representative of the general population to begin with—had (1) continuous counseling and yearlong medical care, as well as (2) frequent monitoring for infection, and (3) surgeries performed in highly sanitary conditions by trained, Western doctors. All of which would be unlikely to replicate at a larger scale in the parts of the world suffering from the worst of the AIDS epidemic. And of course, circumcisions carried out in un-sanitary conditions (that is, the precise conditions that are likelier to hold in those very places) carry a huge risk of transmitting HIV at the interface of open wounds and dirty surgical instruments. So this is a serious point.
What should we conclude? Green et al. get it right: “Before circumcising millions of men in regions with high prevalences of HIV infection, it is important to consider alternatives. A comparison of male circumcision to condom use concluded that supplying free condoms is 95 times more cost effective.”
. . . .
Step 4. This is serious business
The worst part about all of this is not just that the science behind “the circumcision solution” is so shaky, but that the actual implementation of these recommendations—so vociferously pushed-for by the circumcision advocates doing this research — would very likely lead to more HIV infections, not less. The big idea here is “risk compensation” – the subject of an excellent paper by Robert Van Howe and Michelle Storms.
Risk compensation occurs when people believe they have been provided additional protection (wearing safety belts) [such that] they will engage in higher risk behavior (driving faster). As a consequence of the increase in higher risk behavior, the number of targeted events (traffic fatalities) either remains unchanged or [actually] increases.
They argue:
Risk compensation will accompany the circumcision solution in Africa. Circumcision has been promoted as a natural condom, and African men have reported having undergone circumcision in order not to have to continually use condoms. Such a message has been adopted by public health researchers. A recent South African study assessing determinants of demand for circumcision listed “It means that men don’t have [to] use a condom” as a circumcision advantage in the materials they presented to the men they surveyed. [Yet] if circumcision results in lower condom use, the number of HIV infections will increase. [Citations can be found in the original paper.]
In Uganda, as Boyle and Hill uncovered, the Kampala Monitor reported men as saying, “I have heard that if you get circumcised, you cannot catch HIV/AIDS. I don’t have to use a condom.” Commenting on this problem, a Brazilian Health Ministry official stated: “[T]he WHO [World Health Organization] and UN HIV/AIDS program … gives a message of false protection because men might think that being circumcised means that they can have sex without condoms without any risk, which is untrue.”
. . . .
The studies we’ve looked at, claiming to show a benefit of circumcision in reducing transmission of HIV, are paragons of bad design and poor execution; and any real-world roll-out of their procedures would be very difficult to achieve safely and effectively. The likeliest outcome is that HIV infections would actually increase—both through the circumcision surgeries themselves performed in unsanitary conditions, and through the mechanism of risk compensation and other complicating factors of real life. The “circumcision solution” is no solution at all. It is a waste of resources and a potentially fatal threat to public health.
(Read more)
