Generalizing from the armchair

Philosophers frequently make claims about how nonphilosophers think, saying things like “it’s commonsense that,” or “most of us think that,” or “we find it intuitive that…” and so on.

Such remarks amount to empirical claims about human psychology. As a general rule, philosophers should be extremely cautious when making claims about how ordinary people think about philosophical topics.

Here, I outline a handful of reasons why they should be cautious, and discuss such caution specifically in the case of metaethics, where such pronouncements are all too common.

Reason #1: Generalizing is difficult even with data. It’s no easier without it

Given the severe methodological problems in psychology, we're probably not in a good position to make many nontrivial claims about human psychology merely from our conversations. Even dedicated empirical efforts to establish nontrivial claims face enormous challenges in terms of replicability, reproducibility, robustness, generalizability, validity, and so on. In other words, if empirical psychology isn’t able to consistently and reliably tell us what people are like within or across cultures until substantial improvements to the are made, why should we think philosophers relying on introspection and past conversations are going to do any better?

Reason #2: Generalizing based on experience and reflection is probably unreliable in many cases

There are powerful theoretical reasons for suspecting your conversations aren’t a reliable basis for generalizations. Among other reasons:

(a) the conversations you’ve had weren’t from people randomly selected from the populations to which you are generalizing, so they’re unlikely to represent those populations. Philosophers don’t just randomly encounter people; they choose situations where they are disproportionately likely to encounter people with an interest in philosophy, or who differ from the populations they come from in a variety of other ways.

(b) We aren’t passive observers in these interactions, but actively participate in them. As a result, your own views, ways of thinking, ways of framing issues, and so on can casually influence the people you interact with. There are a variety of reasons why this can cause people’s interactions with you to not represent people in general, not the least of which that your interactions with them can change what they say compared to what they’d say if someone else were talking to them (such as e.g., a philosopher with contrary views or a nonphilosopher) .

There are a host of other problems. For instance, our judgments about the conversations we’ve had are likely to be subject to confirmation bias, motivated reasoning, and a variety of other biases and errors. One of the reasons psychologists record participant responses is to have a public record of what was said: our memory and judgments of what was said aren’t reliable enough for making population-level inferences. Such inferences are extremely difficult even when we’ve carefully recorded people’s responses.

Reason #3: Self-selection effects limit what kinds of experiences philosophers have access to

Philosophers may self-select for interactions with people more interested in philosophy. Such conversations are likely to be disproportionately frequent, to last longer, and to be more memorable than conversations with people who don’t want to talk about philosophy. This can cause the populations philosophers are familiar with to be unrepresentative in a very important way: people with an interest in philosophy may systematically differ in how they think about philosophical issues than people who aren’t interested. Yet it’s hard to test because we’d have to get a bunch of people who, by stipulation, don’t want to do philosophy to do philosophy.

I’d be amazed if this weren’t the case. Do we imagine people who want to talk about art or gastronomy aren’t more interested in these topics? And do we suppose that the people we meet in art museums or culinary classes think the same way about art and food as people who we don’t find in these places?

Reason #4: Empirical research in the psychology of metaethics does not support longstanding assumptions about how ordinary people think

In the case of metaethics, we already have empirical evidence on the question. The current and best-designed studies have not found that most people from WEIRD populations endorse moral realism.

Quite the contrary. See e.g., Pölzler and Wright (2020), who found that ~75% of their participants favored antirealist responses, or Davis (2021), who likewise found very high rates of antirealism (including, in contrast to Pölzler and Wright, very high rates of noncognitivism).

Earlier studies likewise found high levels of antirealist responses to a wide variety of moral issues, including Beebe (2015), Sarkissian et al. (2011), and Wright, Grandjean, and McWhite (2013). While some studies found higher rates of realism among children, virtually no studies conducted on adult populations report consistent endorsement of moral realism among participants. At best, what we find is that participants will favor realist responses for some moral issues, and antirealism for others. This might be taken as evidence that these people are realists, but such findings should be interpreted with caution: results may instead be attributable to various confounds and methodological shortcomings with these studies. For instance, the standard method for measuring how people respond to these questions presents people with a moral disagreement, then asks if the two people who disagree can both be correct, or whether at least one must be incorrect, which is known as the disagreement paradigm.

One problem the disagreement paradigm is that if you endorse cultural relativism, or think that two people are indexing their moral claims to the same standards, you could still think at least one must be incorrect, even if you’re not a realist. Sure enough, when Sarkissian et al. (2011) presented participants with a moral disagreement between (a) members of the same culture (b) members of different cultures, and (c) humans and a nonhuman extraterrestrial civilization very different from ours, people were more likely to choose the non-realist response option in (b) than (a), and in (c) than (b), indicating that, antirealist responses increased with the cultural distance between those who disagreed. Should we interpret this as evidence most people are cultural relativists? No, I don’t think so. But what we don’t have is good evidence most people are moral realists. There are many other problems. So many, in fact, that it would be difficult to list them all here without writing a whole essay. Fortunately, someone already did so. See here. To briefly sample the sorts of issues with the disagreement paradigm mentioned by Pölzler, along with ones that strike me as plausible:

1. Participants may provide normative moral judgments rather than metaethical ones

2. Participants may conflate epistemic and metaphysical considerations

3. Participants (or researchers) may conflate realism with universalism, and relativism with sensitivity to context

4. It’s unclear what position (if any) participants take towards truth itself

5. Studies may provide limited response options that don’t exhaust the options available to antirealists

6. Social desirability bias. Participants may not want to choose a realist or antirealist response if they think it would have undesirable social implications

7. Instructions may bias, mislead, or confuse participants. More generally, the general complexity, subtlety, and unfamiliarity of the topic may prompt unintended interpretations of stimuli

8. Treating stimuli as fixed effects when they shouldn’t be

…And there are many more. More generally, there are questions about the validity of these measures. The takeaway, if so, isn’t that we are free to conclude that most people are moral realists, but that, at best, we simply don’t know.

Many philosophers, moral realists and antirealists alike, have assumed that ordinary people endorse moral realism. There is little empirical evidence to support this conclusion. At least in this case, the assumptions among philosophers that they know how ordinary people think are likely to be wrong. It’s beyond the scope of this post, but I believe a strong case could be made that this generalizes, and that philosophers’ judgments are not a reliable indication of how ordinary people think about these issues.

Outliers among outliers among outliers

Questions about how nonphilosophers speak or think are empirical. It is challenging to figure out what (if anything) nonphilosophers think about traditionally philosophical questions using the best tools available to us, spanning quantitative and qualitative analysis, surveys, experiments, and behavioral neuroscience.

For well over a decade now, it has become increasingly apparent that most psychological research relies on samples that aren’t representative of most of the world’s populations. Members of these samples tend to come from WEIRD populations, and WEIRD populations often anchor one or the other extreme end of the distribution along psychological measures (see Henrich, Heine, & Norenzayan, 2010), and, in particular, from college students (most of whom are also from WEIRD populations). The conversations and interactions philosophers engaged in contemporary analytic philosophy are having will likewise tend in many cases to be drawn from these same populations, populations who appear to differ in many ways from other populations. Critically, Henrich et al. (2010)’s examples specifically identify differences in moral psychology. If anything, the study of philosophy may serve to further widen this gap.

Contemporary analytic philosophers tend to study a curated canon of philosophical works from a predominantly Western conception of the world; indeed, analytic philosophy is typically distinguished from continental philosophy in that the latter is identified more with continental Europe, and the former more with the Anglophone philosophical world. And Henrich et al.’s (2010) findings show that it is precisely this population that anchors the furthest end among WEIRD populations. Note, as well, that many psychological studies focus on student populations, and student populations have likewise been shown not to represent non-student populations. Yet philosophers often appeal to their experiences and interactions with students. The experiences and conversations of a philosopher will thus likewise tend to be drawn from populations that may think differently from those populations the philosopher has not interacted with as frequently. And these populations tend to closely overlap with those studied by psychologists.

Why, then, should we suppose that the conversations and experiences philosophers have, which are purportedly sufficient to furnish them with the ability to generalize about how nonphilosophers speak and think, generalize beyond the populations they happen to have interacted with?

If they aren’t justified in generalizing beyond those interactions, then they aren’t in a position to make confident pronouncements about how “people” think or what’s “intuitive,” or “self-evident” or “obvious,” or whatever else, even if their experiences and conversations were reliable for making judgments about the populations they have interacted with: something which I very much doubt.

If philosophers want to start qualifying statements by saying things like:

“Intuitively, according to my recollections of what a majority of students taking my introduction to philosophy courses at a small college in Vermont are inclined to think when I first ask them…”

“I have the general impression most of the people I’ve spoken to at dinner parties agree…”

“I’ve asked this question to three or four of my colleagues, who of course weren’t randomly selected from the population of philosophers, and they all agreed the answer was self-evident…”

“Though I cannot provide much in the way of specifics, I have a kind of rough impression based on conversations and private reflection, that…”

…they’re welcome to do so. But I take it they won’t do this, because such qualifications take all the wind out of the sails of these remarks. The force of claims about what’s “intuitive” or “self-evident” or “obvious” and so on come primarily from the implication that these qualities apply to all or most people, that they are “commonsensical,” that they characterize how all or most people think in a robust way that transcends the idiosyncrasies of particular linguistic communities, or even that they reflect a kind of normative judgment, where anyone competent with the relevant considerations would tend to converge on the same answers as one another, such that there is a “correct” answer regarding the philosophical intuition or judgment in question.

Would moral realists insist moral realism is “the default position,” if their remarks were explicitly confined to <1% of the world’s population? No. And if they said “it’s the default position for people at the sorts of dinner parties I go to…” they might appropriately limit their remarks, but this would no longer pose any threat to moral antirealists since there would no longer be any clear presumption in favor of realism.

Setting all of these objections aside, philosophers who claim we are in a position to make broad and general claims about human psychology based on their personal experiences would need to present convincing arguments for why they are justified in making such claims. After all, not only are general claims about human psychology empirical questions, so too are claims about the reliability of philosophers to make accurate judgments about how nonphilosophers think.

In other words, the only way we could be confident that philosophers don’t need to conduct empirical research to make accurate judgments about how people think is by checking the accuracy of those judgments…by conducting empirical research. We cannot resolve questions about the epistemic capabilities of armchair theorizing about empirical questions by engaging in more armchair theorizing about the epistemic capabilities of armchair theorizing.

Conclusion

Given considerations (1)-(4), I don’t think philosophers are in a good position to generalize about how people think based on their conversations. Such conversations are not representative of people in general (whether from a particular culture or not), were not gathered systematically, and are vulnerable to a variety of substantial biasing influences and potential inferential mistakes.

Philosophical reflection within the analytic tradition may reflect the judgments and conclusions of outliers among outliers among outliers. Efforts to generalize outside of the fairly narrow confines of those with whom philosophers have interacted are probably not justified, for many of the same reasons (plus more) that studies on college students in the United States have turned out not to be a very reliable guide as to how the rest of the world thinks. Philosophers are at an even greater disadvantage, since their judgments aren’t even systematically recorded on spreadsheets that serve as a public repository of what, exactly, people said or did. Instead, when philosophers make judgments about how people think, all of the data they receive passes through an obstacle course composed of virtually every cognitive bias and error there is, from availability to confirmation bias to motivated reasoning to confabulation to selection bias to sunk costs to taking one’s expertise to confer resistance to bias, even when it doesn’t (i.e., a bias blind spot) to the belief bias (i.e., the tendency to judge arguments to be stronger when the conclusions are appealing). Maybe philosophers somehow develop the capacity to navigate this obstacle course, either individually or collectively. If so, how would we know? Presumably, we’d need some independent metric by which we could gauge the success of the methods of contemporary analytic philosophy, and I don’t know of any reasonable candidates other than empirical gauges.

I know of no such methods, and if there are any, philosophers don’t seem to be talking about them, and were any put forward, I’d be willing to bet they’d all argue about it and fail to reach anything even approaching a consensus.

We also face the puzzling fact that philosophers almost never reach any consensus about anything, and questions about whether philosophy makes much, if any progress over the decades or centuries, a regular topic of discussion within the field, a notion that would be preposterous in medicine, engineering, or biology.

Analytic philosophy is supposed to be a discipline that emphasizes rigor and clarity. Within the confines of its (often foggy) methods, it can achieve wonders. Philosophers can carve out conceptual distinctions where others see undifferentiated clumps of words and concepts. Philosophers can distill an argument from a convoluted soup of words and ideas, presenting us with a syllogism as clear as one of those freshly-cleaned sliding glass doors some of us have walked into as though it weren’t there.

I have little doubt there is much to philosophy’s emphasis on clarity and precision that is to its credit. What I don’t grant is that studying contemporary analytic philosophy endows us with deep insights into human psychology. I’m not sure it provides us with much insight on such matters at all. The study of philosophy hardly favors putting the student of philosophy in the sorts of ordinary situations that both shape and reflect how nonphilosophers think, speak, and act. Quite the contrary. The examined life may be one worth living, but it could hardly be said that with our heads in the clouds that our feet still remain firmly planted on the ground.

References

Beebe, J. (2015). The empirical study of folk metaethics. Etyka, 50, 11-28.

Davis, T. (2021). Beyond objectivism: New methods for studying metaethical intuitions. Philosophical Psychology, 34(1), 125-153.

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?. Behavioral and brain sciences, 33(2-3), 61-83.

Nosek, B. A., Hardwicke, T. E., Moshontz, H., Allard, A., Corker, K. S., Dreber, A., ... & Vazire, S. (2022). Replicability, robustness, and reproducibility in psychological science. Annual review of psychology, 73, 719-748.

Pölzler, T., & Wright, J. C. (2020). Anti-realist pluralism: A new approach to folk metaethics. Review of Philosophy and Psychology, 11, 53-82.

Sarkissian, H., Park, J., Tien, D., Wright, J. C., & Knobe, J. (2011). Folk moral relativism. Mind & Language, 26(4), 482-505.

Schimmack, U. (2021). The validation crisis in psychology. Meta-Psychology, 5.

Wright, J. C., Grandjean, P. T., & McWhite, C. B. (2013). The meta-ethical grounding of our moral beliefs: Evidence for meta-ethical pluralism. Philosophical Psychology, 26(3), 336-361.

Yarkoni, T. (2022). The generalizability crisis. Behavioral and Brain Sciences, 45, e1.