Academic journalism is already flooded with ChatGPT apocalypticism, and I don’t intend to add to it. (I am, however, interested in thinking about how generative technology builds on ancient thinking on impersonation and deception!) But it is clear that faculty need to adopt different styles and protocols for tests and assignments, especially in writing-centered humanities courses, in order to incentivize students to learn material thoroughly and to disincentivize thoughtless, “generable” work. One suggestion that keeps coming up in conversation is a return to oral exams, but there’s not too much out there about how to run these oral exams, either as an intellectual exercise or as a practical format, especially for undergraduate-level courses. (Ph.D. orals are a different beast, of course.) I developed a system for efficient, thoughtful oral exams during the first two years of the pandemic, when all my exams were conducted remotely, and perhaps these ideas will be helpful for others who are thinking about doing something similar.
I used this final exam format for Columbia’s Contemporary Civilization course, which in the fall semester has students read from Plato to Rousseau and in the spring from Hume to the contemporary era—it’s a long reading list. The exam is designed to test students’ comprehension of these various readings/authors and also to ask students to make reasonable (and reasoned) judgments about which ideas in those readings they find persuasive. I distributed the instructions for the exam a couple weeks beforehand and asked students to sign up for a 20-minute appointment (over Zoom, but in-person would work just the same). Those time slots add up quickly, but I found it manageable to do 40 or so such exams, especially since there are no stacks of blue books to grade afterward.
The instructions for the first part of the exam—passage identification—read like this:
For the first part of the exam, you will be shown brief passages from three of our texts (spring semester only), and you will need to identify the source/author of each passage with reasonable specificity. I may ask some guiding questions as you talk through the texts. Each passage will be worth 10 points for a total of 30 points.
I set aside 10 minutes for this first part. I had selected 20 emblematic passages from our readings, and I asked students to pick their “three lucky numbers” from one to twenty. (This helped lighten the mood for all of us who were entirely sick of Zoom at this point.) I would then flash—via Screen Share—a passage like the example below:
Passage #11: But the state of the case is in every respect the reverse of this. In the first place, the opinion in favour of the present system, which entirely subordinates the weaker sex to the stronger, rests upon theory only; for there never has been trial made of any other: so that experience, in the sense in which it is vulgarly opposed to theory, cannot be pretended to have pronounced any verdict. And in the second place, the adoption of this system of inequality never was the result of deliberation or forethought, or any social ideas, or any notion whatever of what conduced to the benefit of humanity or the good order of society.
As students read the passage, I would remind them that I was less interested in blurting out the correct identification of the passage and much more interested in what sticks out from the passage. So rather than simply say “It’s The Subjection of Women by John Stuart Mill and Harriet Taylor,” I would want students to point out Mill’s suspicion of using “theory only” to guide social progress (perhaps recalling the “experiments in living” from On Liberty) and also to underscore his attention to “the benefit of humanity” as a sign of his interest in utilitarian thought. If a student wasn’t quite getting the right cues from the passage, I might pull out of one those phrases and ask what it meant and which of our thinkers it might be the most appealing to. I’d assign 5 points for the correct identification of the author/text and another 5 points for a reasonable explanation of important terms and ideas in the prose. We’d do this same exercise for two other passages for the first part (Passage Identification) of the exam.
The second part of the exam is designed to replace a blue-book essay. That is, I wanted to test students’ abilities to make a real argument about a position they hold, and I wanted them to use evidence from our readings to back up that argument. When I distributed my exam instructions a couple weeks beforehand, I asked my students to read three passages/prompts carefully and to prepare their own arguments about those passages, one of which they would need to speak about during our exam. The instructions read as follows:
Before your oral exam, you should read through the three passages below and consider how our authors and ideas might relate to these brief selections. During your exam, I will ask you present your thoughts on just one of the three passages (of my choosing). Following the structure of the midterm, you’ll first need to describe three connections (either similarities or differences) between the ideas presented in the passage and an author/text in our course readings up to this point. To be clear, you will need to name three specific authors/texts. Next, you’ll need to decide whether you agree or disagree with the passage’s central claim, referencing our texts as support for your thinking. In this second part, you will need to take a specific position either for or against the provided passage. This half of the exam is worth 30 points, equal in weight to the entire first half.
Here’s one of the passages I used in the second year:
Passage #2: [S]o barbarous is the anthropological value system to which contemporary American social science seems to be geared that so far as the technicians who survey Negro communities are concerned, people without affluence and power are only creature-like beings whose humanity is measured in terms of their potential to accumulate material goods and exercise force with arrogance.
Alas, not even the most fundamental human value that democratic societies are specifically designed to guarantee seems to count for very much once such technicians become involved with Negroes. On the contrary, far from revealing any significant preoccupation with or even appreciation for personal freedom and self-realization in any intrinsic sense, the technicians now proceed in an alarming number of instances as if statistical measurements of central tendencies […] have become a means of justifying an ever-increasing standardization, regimentation, and conformity. In so doing, they tend to condemn the very elements in U.S. Negro life style that other non-totalitarian cultures seek and celebrate: its orientation to elastic individuality, for one, and its esthetic receptivity, and its unique blend of warmth, sensitivity, nonsense, vitality, and elegance.
The Omni-Americans: Some Alternatives to the Folklore of White Supremacy by Albert Murray (1970)
If I chose this passage during the exam, I would first ask my student to summarize the claim in Murray’s prose. I might ask some guiding questions if the student had some blind spots, but once we had a workable idea of Murray’s claim, I’d ask the student to give me a response—is Murray on to something here, is it all wrong-headed? What should we think about his argument? Students were expected to bring in their own fully formed ideas and a few texts to use as supporting points (or even anti-examples). Here, students might cite Plato’s Republic as a dystopia/utopia built on “standardization, regimentation, and conformity,” and they might draw in Tocqueville’s concept of the psychological “tyranny of the majority” as a foil to Murray’s celebration of “elastic individuality.” (Nietzsche could be good, too.) Students might see in Murray a critique of Marx (or Smith) in the rejection of “material goods” as a metric of historical development. You get the idea: the students need to have thought about how the passage here sits among the smorgasbord of readings from our semester. The students’ initial comments would almost always springboard into a rich conversation about some of the values and arguments they found appealing from our texts over the year.
I’d assign 7 points per thoughtful connection to another text (for a total of 21 points), and 9 points for a clear argument for/against/about the passage. The ability to make this kind of argument was a central aim of the course, and it was one that we had practiced throughout the year.
I’m happy to share the remaining passages with other faculty who want to use a format like this one—just e-mail. I used this format four times at Columbia and once at Minnesota, and it has always worked really well with my undergraduates. It incentivizes exactly the kinds of reflective thinking we want to see in our students’ writing. And over and above that, it seems like it could be a good format for obviating any AI-related problems with academic integrity and stale, generated responses.