The Legal Beat

It Turns Out ChatGPT Wouldn’t Be The Best Law Student…
Posted on Thursday January 26, 2023

Alone, ChatGPT would be pretty mediocre law student. The bigger potential for the profession here is that a lawyer could use ChatGPT to produce a rough first draft and just make their practice that much more effective.

— Professor Jonathan Choi of the University of Minnesota Law School, in comments given to Reuters, on how the ChatGPT AI program would perform in law school. In a recent study, Choi collaborated with professors Kristin Hickman, Amy Monahan, and Daniel Schwarcz to see how ChatGPT would generate answers on four Minnesota Law exams that they blindly graded along with the exams of real students. On average, ChatGPT performed at the level of a C+ student after completing 95 multiple choice questions and 12 essay questions. ChatGPT performed the best in Constitutional Law (B), the second best in Employee Benefits (B-), and the worst in Torts (C-) and Tax (C-).

Staci Zaretsky is a senior editor at Above the Law, where she’s worked since 2011. She’d love to hear from you, so please feel free to email her with any tips, questions, comments, or critiques. You can follow her on Twitter or connect with her on LinkedIn.

ADDITIONAL STORIES: