Study Mode, Socratic AI, and the Future of Legal Education

OpenAI announced today a feature called "Study Mode" available to users of ChatGPT. Basically, its a Socratic Professor. Instead of answering questions all at once, it helps you answer the question for yourself. This new application of AI it reinforces just how urgently and rigorously academics, including legal academics, have to think about their role. In some sense, Study Mode doesn't do much more than AI could do last week. You could indeed instruct it to assume the role of a professor of whatever style you preferred. But what Study Mode does is to build that capability right into ChatGPT and lower to zero any barrier to using it. It is an in your face declaration that AI can do an awful lot of what professors in law and elsewhere have done for decades.

Let's take an example from this afternoon where I assume the role of a fairly strong but confused student in Constitutional Law battling the complexities of presidential removal powers. All it took was to click "Study and learn" in the Tools part of the ChatGPT prompt. Here's a link to the entire conversation.

Here's how the conversation began. Notice that the AI intelligently tries to figure out where the student is so they can help them at the appropriate level.

The AI follows up with questions that draw out the complexities involved and gives positive feedback for (immodestly) an intelligent response.

We keep going and ultimately the AI gives me hypos, just like a real law professor might in probing a student's understanding.

At the end, I wanted to see if the AI was up to date on all the recent issues surrounding presidential removal power. It was.

At the end of the day, ChatGPT did a perfectly respectable job in helping the student understand the basics of presidential removal powers. Could I have done it better? Possibly, though I am frankly not sure. And, unlike ChatGPT with its accursed patience and seeming lack of other responsibilities, I do not have the time to "meet where they are" every student in an 80-person class and individually move them forward. And I, like other professors, try to regularize contact hours with students. I don't generally respond immediately to emails at 11 p.m. on a Sunday about the dormant commerce clause. (Though sometimes I do).

In short, ChatGPT does an awful lot of work traditionally done in classes and does it quite well. And lest you think that it only did so well because presidential removal powers is a fairly well plowed field, I next tried with something more esoteric, the duty to settle under Texas insurance law. I picked the topic because I know a fair amount about it and would be able to spot errors. Suffice it to say, ChatGPT did splendidly. It understood the doctrine, explained the rationale, was aware of law and economics challenges to the doctrine, and interacted in a gentle Socratic way with me (posing again as a confused student).

So the question is, what do we professors do now? We could bore the students to tears and waste time by teaching the same material that ChatGPT Study Mode could handle. And we could justify that course of action by seizing upon the occasional hallucination that ChatGPT will surely make and claim that AI just isn't good enough to teach our students. We would sub silentio excuse, of course, those times when we made a mistake while teaching complex material. And in rejecting AI, we would ignore the fact that it is current doubling in its capabilities about every semester.

Or we could accept the new technology and look for alternatives. For example, we could "flip" the classroom and make our job both to come up with good starting points for student interactions with ChatGPT Study Mode and then make class more a place for discussion. Will students have the discipline to learn and thrive in this new world? Will the gentle ChatGPT professor be insufficiently scary to motivate the students in the same way that the Kingsfield Socratic method was alleged to do in the now-antique Paper Chase movie. (assuming anyone really does that schtick anymore)? I'm not sure. On the other hand, what Professor Kingsfield describes as the Socratic method appears to be almost exactly what ChatGPT Study Mode does.

"Through my questions you learn to teach yourselves. Through this method of questioning, answering, questioning, answering, we seek to develop in you the ability to analyze that vast complex of facts that constitute the relations of members in a given society."

Will reliance on ChatGPT, however, with its chipper tone and limitless ability to answer, however, reinforce the delusion of certainty and closure Professor Kingsfield warns about?

"At times you may feel that you have found the correct answer. I assure you that this is a total delusion on your part. You will never find the correct, absolute, and final answer. In my classroom, there is always another question, another question to follow your answer. ... We do brain surgery here. You teach yourself the law but I train your mind."

Perhaps. On the other hand, the bar exam is devoted to the concept that there are definitive answers to many legal questions and, in many instances, the bar examiners are right.

I suspect that ChatGPT’s Study Mode—and similar tools soon to follow from other AI vendors—will swiftly become the inexpensive yet invaluable partner of students learning the law. We professors may still retain the role of training their minds in class, or perhaps inspiring them by the force of our presence. Maybe. But the legal academy, and American universities more broadly, must begin thinking at more than the customary academic pace about AI’s revolutionary impact on education—a transformation that today’s Study Mode makes pellucid.

Or be left in the dust.