Scientists Have Unwittingly Revealed that the Obergefell Decision Did Nothing to Diminish Sexual Minority Distress

In a study first appearing online in JAMA Psychiatry on May 23, five public health scholars report that “state laws permitting denial of services to same-sex couples” are associated with mental distress in sexual minority adults who live in those states in a way not visible in surrounding states. The study received a significant media boost in the days immediately following the June 4 decision of the Supreme Court in favor of Jack Phillips, the Colorado baker whose Christian convictions shaped his decision to avoid creating and selling wedding cakes celebrating same-sex nuptials.  

The implication of the JAMA study is that the Court has just opened the way for “anti-gay” conscience rights that—according to the study’s authors—have the documented potential to erode the mental health of sexual minorities. It certainly is an intriguing claim and possibility, and the study deserves a closer look.  

When we give it that closer look, however, it becomes obvious why the study’s title begins with the word “association,” that is, connection or relationship—because it is far from clear whether it is a valid, reliable link. While the title makes that clear the authors are not equating correlation with causation, the media narrative suggests otherwise. (That’s how this partnership has long worked.) 

What the Study Says 

Here’s the story. Utah, Michigan, and North Carolina each passed “laws permitting denial of services to same-sex couples” in the spring of 2015. What sort of denial of services? Are we talking about refusal of service in a restaurant or denial of medical treatment—two examples that one Indiana University sociologist cited when asked about his take on the new study? Nothing of the sort. Two of the statutes enabled public officials to opt out of licensing or performing same-sex marriages—which did not threaten the ultimate delivery of either service—while the other permitted private adoption agencies to prefer opposite-sex couples in placements, if they so wished.  

The study’s key finding is that 22 percent of sexual minorities in Utah, Michigan, and North Carolina reported mental distress in 2014, but 33 percent did so in 2016, a notable and significant uptick. Why? That is the pivotal question. Even in the six nearby comparison states—where no such laws were recently enacted—distress climbed, but only modestly. The authors document a statistically significant “multiplicative interaction effect” between sexual minority status and laws permitting denial of services to same-sex couples, meaning mental distress worsened among this group, but only if they lived in one of those three states. 

The three states all passed their regulations before the Obergefell decision—albeit perhaps in an anticipatory manner—not after. The authors are thereby asking the reader to believe that the triumph of Obergefell was so dampened in at least three states (by the fact that there was an opt-out clause enabling conscientious objecting clerks and adoption agencies to quietly pass on participating) that mental distress rose precipitously for sexual minorities there.  

Applying Common Sense 

I will not—indeed, without having analyzed the data myself I cannot—contest the veracity of the finding. (Although it’s prudent to do so in this domain of study.) What I am in search of is a sensible explanation for why sexual minorities’ mental distress would worsen in the year after Obergefell—the mother of all victories—rather than improve? Indeed, it would appear there was no Obergefell benefit for the mental health of sexual minorities in these data, regardless of where they lived. Why would that happen in this new era of constitutional protection of same-sex civil marriage, and of poll numbers signaling surging popular support for same-sex marriage (new data that were released the same day as the JAMA Psychiatry study)? That is surprising. Perhaps what prompted Obergefell, the lament of unfair denial of access to civil marriage, was never a key source of mental distress.  

The case for making social-scientific arguments about valid association between two variables—in this case passing “denial of service” laws and subsequent increased psychological distress—involves two very basic aspects: first, it must not contradict actual observation. In this case, I have to presume the authors are telling us the truth about the empirical association.  

Second, it must make logical sense. That’s where this study wavers. The first author admits that “it is unlikely most of them (that is, sexual minorities) directly experienced service denials.” In this case, the logic is that if any portion of the law—or if people around me—don’t wish to participate in some narrow aspect of my wishes, it will contribute directly to my poorer mental health, even while I know that wide popular support for my general interests is elevated and growing by the year. It doesn’t make sense.  

As the authors note, “In the 2012 to 2014 dataset, trends in mental distress were not statistically significantly different in states that implemented laws permitting denial of services to same-sex couples compared with control states.” Why not? In 2014, many states’ constitutions still forbade same-sex couples to marry. Isn’t that a more discouraging thought? How would experiencing the Obergefell decision—tarnished only by a few state opt-out clauses for a minuscule number of actors—subsequently depress sexual minorities more? 

The Devil in the Details 

The devil may be in the details. There is a reason why this study comes with so many “supplemental” online tables, which are typically added at the request of blind reviewers and editors. The author’s approach to data analysis appears akin to data “mining,” searching for findings by including lots of variables, including complex “multiplicative interaction effects” that make straightforward interpretations challenging.  

For example, there are thirteen different interaction effects in the one model that visibly generates the study’s key claim. I typically fret over including and interpreting even one interaction effect correctly—they are not simple. Including twelve others in a model that features dozens of predictor variables violates my (subjective) preference for parsimony in model-building.  

This also raises concern that the key interaction effect—the one that generates the story—is sensitive to the inclusion of other control measures. But the reader (of the supplemental tables) is only treated to one model rather than the series of models that typically characterizes a prudent way of constructing evidence for a claim, one that builds readers’ confidence that what you have discovered is robust and firm. Unfortunately, in the domain of study on matters of sex and sexuality, this is rare form. My frustration with data analyses in this domain has long concerned not the data themselves but the measurement and analytic decisions that are made. It’s not the science; it’s the scientists. 

For another example, why does the published version of this study lean on linear regression models that, the authors’ excuses aside, are not standard for the type of outcome they feature? Indeed, in the supplemental online data, the more appropriate (in my mind) analytic approach yielded weaker effects. Why draw the measurement line for poor mental health at saying “not good” in “14 or more of the past 30 days”? I would think those who claimed twenty-five days out of the past thirty are categorically in tougher shape than those who said fifteen days or those who said eight.  

As in other studies I’ve criticized, there are questionable measurement decisions for which I am left to wonder about the underlying logic. Is it because this threshold reveals the desired effect, and not others? Basically, are the authors acting in good faith, or fishing for the model that enables them to tell a particular narrative? I don’t know.  

What’s sensible about this study? Several aspects, actually. Sample size is never a problem. These are massive data collection efforts, and confidence in the CDC’s data runs high. No objections here. But that makes the decision to lump together all 4,656 gay, lesbian, bisexual, and transgender respondents all the more difficult to defend. Why voluntarily lose information and precision? If you have the sample size to distinguish between different groups, do it.  

Other findings, mostly discerned in the supplemental data, make sense: women report higher distress than men, the poor more than the wealthier, the unemployed more than the employed, and the unmarried more than the married. Those are each consistent with previous studies.  

In the end, we’re left to speculate about a truly odd result. Is it possible that mental distress doesn’t get better for sexual minorities? Or—as the logic of this study implies—is it that it won’t get better until conscientious objection is no longer permitted?