Training paradigms and Spontaneous Theorizing in Folk Metaethics Research

1.0 Introduction

Hans P. Niemand posted a critique of a dilemma I’ve posed about training paradigms in the psychology of metaethics. I encourage you to read that critique prior to reading this post (and do consider subscribing).

The primary goal of research in the psychology of metaethics is to determine whether nonphilosophers are moral realists or antirealists (and if so, what kind of realist or antirealist). This research has faced significant methodological challenges (see here and here for relevant articles). Here’s the abstract of my paper with David Moss addressing this issue:

Recent research on the metaethical beliefs of ordinary people appears to show that they are metaethical pluralists that adopt different metaethical standards for different moral judgments. Yet the methods used to evaluate folk metaethical belief rely on the assumption that participants interpret what they are asked in metaethical terms. We argue that most participants do not interpret questions designed to elicit metaethical beliefs in metaethical terms, or at least not in the way researchers intend. As a result, existing methods are not reliable measures of metaethical belief. We end by discussing the implications of our account for the philosophical and practical implications of research on the psychology of metaethics.

In my later work, I argue that the reason participants reliably fail to interpret questions as intended is because they have no metaethical stances or commitments in the first place. That is, they struggle to interpret these questions as intended because they don’t possess the relevant concepts and don’t have positions on them, at least prior to participating in the studies where they are exposed to metaethical realism and antirealism.

One solution to the methodological problems with earlier research in the psychology of metaethics is to teach participants about the relevant metaethical positions, ensure they adequately comprehend those positions, and then measure their metaethical views. I call these methods training paradigms. I argue that training paradigms face an insurmountable dilemma:

1. If they fail to induce adequate comprehension, the measures they employ remain invalid. That is, if participants still do not interpret stimuli as intended then their responses cannot reflect their metaethical positions. For comparison, if you insist people fill out a survey in a language they don’t understand at all, their responses will not be very informative.

2. If they succeed at inducing adequate comprehension, this may cause participants to form a metaethical position. If this occurs, then such participants are now psychologically unlike the population they came from. As a result, findings in such studies cannot generalize to the populations of interest.

In other words, training paradigms either fail on their own terms or succeed at the cost of a nearly total loss of external validity. I will develop on my dilemma more below, but I will first turn to Hans’s critique.

2.0 The critique

Hans begins with the following remark:

Lance Bush has proposed a dilemma for the study of folk metaethics. He claims it is impossible, or at least extremely hard, to empirically study the metaethical intuitions of non-philosophers [...]

One initial nitpick: I consciously and explicitly avoid talk of “intuitions” in my discussion on this topic (at least in most contexts; I may go along with such language in public discussions even though I dislike it). I don’t think “intuitions” are a legitimate psychological category and instead employ proprietary terminology (“stances,” and “commitments”, which I define elsewhere, but which roughly correspond to explicit positions and implicit facts about how people think or speak independent of their explicit commitments, respectively). As such, I wouldn’t describe my position to be that it’s hard to study people’s “metaethical intuitions,” since I have issues with the term intuitions.

This isn’t a big deal, though, but it’s worth flagging because my beef with intuition-talk is a major feature of my broader metaphilosophy. In any case, Hans goes on to do a fairly good job of depicting one dilemma (there are nested dilemmas within dilemmas on my view, which I’ll get to):

1. If you naively ask them questions about their views, they pretty reliably interpret the questions in ways the researchers didn't intend.

2. If you train them to understand the concepts and distinctions you're asking about (as e.g. Pölzler and Wright do in their "Anti-Realist Pluralism" paper), then they are no longer "non-philosophers." These “training paradigms” turn participants into "baby philosophers," and their answers reflect nascent theorizing about the questions, not untutored intuitions.

This seems fine. I’m happy to give that a thumbs up.

Hans appears to accept the first horn of the dilemma, but takes issue with the second one. This is where I think the objection goes astray. Hans says:

I'm less convinced by the second horn. "Non-philosopher" is not very precise, and whether study participants in training paradigms count as "non-philosophers" in the relevant sense depends on why we thought the intuitions of "non-philosophers," as opposed to the intuitions of philosophers, would be methodologically important for resolving philosophical questions.

Whether “nonphilosopher” is precise is, as I will show, irrelevant to the problem I raise for training paradigms. And yes, it is imprecise in an entirely unavoidable way: there is no precise cut-off between “philosophers” and “nonphilosophers,” so the lack of precision is an inevitable byproduct of the way reality works. There aren’t always (if ever) sharp cut-offs between different categories. I don’t know if criticism was implied in the throwaway line about the notion of a nonphilosopher being “imprecise,” but it wouldn’t be a compelling criticism if it were since nothing about my dilemma relies on a sharp distinction between philosophers and nonphilosophers (which is convenient, because I don’t think there is such a sharp distinction).

The bigger issue with Hans’s objection is the latter part of this quote:

[...] and whether study participants in training paradigms count as "non-philosophers" in the relevant sense depends on why we thought the intuitions of "non-philosophers," as opposed to the intuitions of philosophers, would be methodologically important for resolving philosophical questions.

…here is where the objection gets very strange. My objection to training paradigms is rooted in methodological concerns related to measurement. It does not turn on any precise distinction between philosophers and nonphilosophers, and, more importantly, it doesn’t turn on the methodological relevance of the intuitions of nonphilosophers to resolving philosophical questions (at least not in any direct way). So my objections simply do not depend on why we thought the intuitions of nonphilosophers are methodologically important to resolving philosophical questions.

I said “at least not in any direct way” because there is one sense in which my claims are relevant to philosophical disputes: if I am correct, it is not the case that most nonphilosophers or “ordinary people” endorse moral realism or speak, think, or act like moral realists. If I am correct, this undermines claims that there is a presumption in favor of moral realism because it is a “commonsense” or “intuitive” view widely held by nonphilosophers or is implicit in the way people are generally disposed to speak and/or think. In that respect, though, my dilemma does just fine, because it centers on validity of training paradigms: that is, it centers on whether training paradigms successfully measure what they intend to measure (validity in a more general sense), and, if so, whether such measures can generalize to populations outside the context of the study (external validity). Since the dilemma I propose maintains that training paradigms are either invalid in that they fail to measure people’s metaethical views or lacks external validity in that it does succeed but doesn’t allow for generalizations outside the study, if I am correct about these points, this would threaten the utility of such studies in resolving questions about whether moral realism is “commonsense.” However, the objections Hans raises have little to do with this.

Hans continues:

So, if you're already in the business of thinking that intuitions play any important role in philosophical methodology, why not just use the intuitions of philosophers? I can think of a few worries one might have:

As noted, this is already going to be a problem for me due to the reliance on intuition-talk, but we can bracket that now and consider the points Hans raises:

1. A philosopher's intuitions might be distorted because they are already committed to a theory about a question. For instance, a utilitarian might find it intuitive that it's not wrong to push the fat man man with a large backpack off the bridge in the trolley problem, simply because this is what his theory implies and so to have the opposite intuition would produce cognitive dissonance, while someone who has not done much ethical theorizing before might not find it intuitive. So the worry is that the utilitarian's intuition is biased by his theory.

2. A philosopher's intuitions might be distorted because they have been taught or inducted into having certain intuitions (for example, that a life in the experience machine is worse than an experientially-identical life lived in the real world). So again, the worry is that the philosopher's intuition is biased, this time by the culture or training of contemporary academic (especially analytic) philosophy.

3. A philosopher's intuitions might be, not distorted, but simply biased, because people with certain intuitions are simply more likely to go into (contemporary, academic, analytic) philosophy, or to stay there. People with different intuitions might be less interested in staying in philosophy for a variety of reasons.

Hans then adds:

So, do these worries apply to the study participants in training paradigms? I think they do not, at least not in virtue of the training they get during the study.

Alright. But what does this have to do with my dilemma?

Not much. My objections to training paradigms don’t turn on (1)-(3). Training paradigms may bias participants by teaching or inducting them into having “certain intuitions” (e.g., realism or antirealism). This is a legitimate worry with such studies. But it is not my primary concern. Even if we assume the training paradigms don't bias participants towards realism or antirealism at all, this is entirely irrelevant to my objection, which is that participants had no view one way or another prior to participating in the study. The problem with such studies is that at best they cause people who didn’t have a metaethical position to have a metaethical position.

Hans adds:

Certainly, participants may already have gone through a philosophy degree program, or already have thought about these questions independently in a way that makes them subject to the worry about cognitive dissonance.

Ideally, such participants would be excluded from the analysis since they aren’t genuine members of the target population. Hans claims:

But, assuming participants that have not gone through a philosophy degree program, have not done extensive independent reading in philosophy, and have not thought much about these questions before the study, does the training they get during the study make them subject to the worries above? I don't think it does [...]

…Okay. Let’s suppose I agree for the sake of argument. That…has absolutely no impact on my dilemma at all. Hans appears to have made assumptions about the dialectical role of my dilemma that are completely off the mark.

3.0 The purpose of my dilemma

So, why do Hans’s objections not work? For the simple reason that my dilemma serves an altogether different purpose. Here’s essentially what I argue:

1. Nonphilosophers do not have determinate metaethical positions. That is, ordinary moral judgment and action doesn’t typically involve any explicit or implicit stance towards or commitment to moral realism/antirealism.

2. Studies designed to elicit the metaethical stances of nonphilosophers cannot succeed because there isn’t anything to measure: they don’t have views on these issues, any more than people with no training in quantum mechanics will have views on whether the Copenhagen or Many Worlds interpretation is correct.

3. Studies that attempt to circumvent standard methodological problems by attempting to elicit people’s metaethical views by training people aren’t going to work because they either (a) provide inadequate training, in which case the studies remain invalid or (b) they provide adequate training, which entails that whatever they did, it was successful at inducing participants to interpret stimuli as intended.

4. In the latter case (b), participants will either (i) still not have a determinate view, in which case there’s nothing to measure or (ii) report a determinate view.

5. In the former case, (i) the measure continues to fail to measure whether the participant is a realist or antirealist. In the latter case, (ii), the participant will either (x) report a view they don’t actually hold, in which case the measure is invalid or (y) report a view they now hold but did not hold prior to participating in the study, in which case the participant’s psychology has been changed in such a way that they don’t represent the population they were drawn from.

Point (5)(y) is the key point here. What renders training paradigms invalid is that if, ex hypothesi, people don’t have determinate metaethical stances/commitments, successfully inducing them to adopt and express such positions by participating in the study entails that it is the study itself that caused them to have the position in question.

This results in the study lacking external validity. One cannot infer from the fact that, e.g., 70% of the participants in the study endorse moral antirealism that approximately 70% of the population they came from endorse antirealism, because the participants in the study only began to endorse moral antirealism because they were caused to endorse it by participating in the study. Since the population they came from did not participate in the study, they could not be caused to endorse antirealism (again, on the assumption that they don’t have a view if they don’t participate in the study and thus nothing else caused them to have a metaethical stance or commitment).

As a simple comparison: suppose we discover a population of people that have never heard of Christianity. They neither endorse nor reject it. Now suppose we bring 1000 people from this population to a church where they listen to a philosopher tell them all about Christianity in a neutral and unbiased way (that is, the philosopher carefully attempts to neither prompt them to accept or reject Christianity. Let’s suppose they succeed in being neutral). Now we ask the people whether they:

1. Accept Christianity as true.

2. Reject Christianity as false.

We find that 20% of the population accepts Christianity as true, and 80% reject Christianity as false. Would this allow us to infer that ~20% of the population of the island are Christians, and the other 80% reject Christianity as false? No. That would be absurd. The rest of the island neither accepts nor rejects Christianity.

I am making the same argument regarding moral realism/antirealism: people simply don’t have any views one way or another until they understand and consider competing positions. Studies that teach them about the positions that insist they pick one either fail to get them to understand the positions, force them to pick a position they don’t actually endorse, or succeed at getting them to consider and sincerely pick a position. In the former two cases, the study isn’t valid. In the latter case, it lacks external validity (i.e., it doesn’t generalize to the target population).

So my dilemma is essentially this: if nonphilosophers don’t have determinate metaethical stances or commitments, training paradigms are either: (a) fail to measure people’s metaethical views or (b) lack external validity.

4.0 Preexisting intuitions

Several of Hans’s subsequent remarks reveal a deeper misunderstanding about what I am arguing for and why (and for this section, I’ll employ intuition-talk despite rejecting it for ease of exposition). For instance, Hans says:

The short amount of training a participant gets during the course of a study shouldn't be enough to create the kind of cultural immersion necessary to reshape someone's intuitions; I would think that would take longer exposure over years, either through a degree program or an extended amount of independent reading.

I am not proposing that these studies reshape intuitions. This implies that they held metaethical intuitions prior to participating in the study (or, more specifically, intuitions about the realism/antirealism dispute). Yet this is precisely what I am ruling out ex hypothesi. I believe Hans may be operating under the mistaken assumption that everyone already has metaethical intuitions and the goal of these studies is to find out what they are. My entire argument is predicated on a downstream assumption that this is false, and that because this is false, it renders training paradigms useless. Hans continues:

So, at least in virtue of the training they get during the study, participants' intuitions aren't subject to the biasing influences above. So, if the reason you were interested in the intuitions of non-philosophers was to get access to intuitions that were not subject to these biasing influences, then the training given in training paradigms doesn't pose an obstacle to that, and to that extent training paradigms aren't subject to Lance's worry about turning participants into "baby philosophers."

This objection simply does not work. My dilemma has nothing to do with the kinds of biasing influences that Hans focuses on. And Hans again hints at misunderstanding my position by stating:

[...] if the reason you were interested in the intuitions of non-philosophers was to get access to intuitions that were not subject to these biasing influences [...]

Again, my dilemma rests on the contention that nonphilosophers don’t have intuitions of the relevant kind for you to “get access to.” I am not arguing that nonphilosophers have realist or antirealist intuitions, and that training paradigms can distort or bias them; I am arguing that they don’t have realist or antirealist intuitions at all, so causing a group of people to form such intuitions can’t tell us anything about which intuitions people who you didn’t cause to form intuitions have, because they don’t have any.

Strangely, Hans goes on to argue that if we don’t train people, they may not interpret stimuli as intended:

[...] it seems to me that intuitions about a question would only be probative if the person having the intuition actually understands the question. Otherwise, they're having an intuition about a different question, and it's not clear why their intuitions about that question would be relevant to the truth of answers to the original question.

…I agree. This is a central pillar of the dilemma I pose. Call this the mismeasure outcome. Yet Hans goes on to conclude that:

[...] if we don't "train" study participants to understand the questions the way we intend, then it's not clear why their intuitions would be relevant.

This is precisely the point of the dilemma! If we don’t train them, we’re stuck with the mismeasure outcome. If we do train them, then we will either succeed or fail. If we fail, we’re still stuck with the mismeasure outcome. If we succeed, then I argue that we are causing them to have the views in question, not discovering what their views were prior to participating in the study. When this occurs, our findings can no longer tell us about how people outside the context of the study think. Call this the spontaneous theorizing outcome. My dilemma thus holds that all attempts to measure what isn’t there either are doomed to failure, but can create the illusory appearance of success. This is because the goal of these studies is to tell us whether nonphilosophers out in the world are realists or antirealists. You cannot successfully do this by gathering up such people and then causing them to become realists or antirealists.

Hans goes on:

So: The point of getting the intuitions of "non-philosophers" is not to get the intuitions of people who don't understand the questions we're asking them. It's to get the intuitions of people whose intuitions are not subject to the kinds of biasing influences that (professional) philosophers' intuitions are subject to.

…I’m arguing that they don’t have “intuitions” about these issues. This claim about the point of these studies presupposes the very thing I am arguing against.

Hans then says this:

Lance argues that training paradigms may produce "spontaneous theorizing"—that is, participants might be caused to form their views by participating in the study—and that if so, we can't use their responses as evidence of what "ordinary" people, who haven't been trained in the relevant concepts and distinctions, think about the questions. I think this is true, but unimportant.

…Huh? But…that’s my whole point. My whole point is that these studies don’t tell us what people who “haven’t been trained in the relevant concepts and distinctions, think about the questions.” Of course it’s relevant to that! That is the point!

Hans says:

Why should we care about what people who haven't been trained in the relevant concepts and distinctions think about these questions?

…because the question I am addressing is what nonphilosophers outside the context of these studies think about these questions. And what are nonphilosophers outside the context of training paradigms? They are, quite literally, by definition, people who haven't been trained in the relevant concepts and distinctions. In other words, Hans is asking why we should care about:

what people who haven't been trained in the relevant concepts and distinctions think about these questions

When my entire dilemma is specifically intended to address exactly this question, i.e., what people who haven't been trained in the relevant concepts and distinctions think.

This is a bit like if I were arguing about whether “most people like pineapple on pizza” and Hans asked:

But why should we care about whether most people like pineapple on pizza?

This wouldn’t make any sense as an objection. It’s as if Hans thinks the question of what people without training in the distinctions think is entirely incidental to my argument, when it is quite literally the entire point of the argument.

Hans’s next remark leaves me incredulous:

Actually, maybe it's better to say that people who haven't been trained in the relevant concepts and distinctions could not have views about these questions: the questions are framed in terms of the relevant concepts and distinctions (and so must any answers to the questions be), so anyone without the relevant concepts and distinctions literally could not have views on those questions, in the same way that someone who doesn’t know what a “vector” in the mathematical sense is could not have views on linear algebra.

…my dissertation is a defense of exactly this claim. Hans doesn’t even seem to be unaware of this, because the very next remark is:

(As an aside, this seems like a solid argument in favor of Lance's "folk metaethical indeterminacy"—the folk do not have determinate views on metaethical questions, because they don’t understand the relevant concepts and distinctions.)

…Yes…why do you think this appeared in a dissertation defending folk metaethical indeterminacy? (That’s a rhetorical question). Hans goes on:

But if our motivations for being interested in the views of "non-philosophers" are what I've been describing, we shouldn't be interested in the views of untrained non-philosophers (since those non-philosophers couldn't have views about the questions, and even if they could, it's not clear why they'd be reliable).

…Maybe so, but that has nothing to do with my dilemma. Hans continues:

Rather, we should be interested in the views of non-philosophers who have been trained in the relevant concepts and distinctions (Lance calls these "quasi-philosophers"): these are the people who can have views about the questions, but whose intuitions won't be subject to the kind of biasing influences I mentioned earlier.

Interested for what purpose? My goal in bringing up the dilemma was to assess the validity of measures designed to tell us about the metaethical views of the population “nonphilosophers.” Insofar as quasi-realists are not members of that population, facts about what intuitions they have wouldn’t be an appropriate basis for making inferences about the intuitions of nonphilosophers. For comparison, if we wanted to know about the cooking skills of people with no culinary training, we certainly wouldn’t want to study professional chefs. But we also wouldn’t want to gather up a bunch of people with no culinary training, give them a crash course in the culinary arts, and then evaluate their cooking skills.

Hans concludes:

So while "spontaneous theorizing" probably happens, I don't think it's a problem; I think it's exactly what we want.

It’s not what we want if we want to make inferences about what people who haven’t theorized (spontaneously or otherwise) think. Which is the entire point of my dilemma.

5.0 Conclusion

I remain puzzled about Hans’s objection. I appreciate when someone writes a friendly critique of my views, but I think Hans has misunderstood the point of my objection. Hans seems to be interested in some other project where we focus on the intuitions of people who understand the concepts in question for some philosophical end. But that simply isn’t what my dilemma is about at all.

Unfortunately, I do not think this critique is successful because it does not appear to engage or offer a critical response to the point I was making.