The Oklahoma Bar Journal August 2023

first draft that needs your careful editing. But legal research tools with appropriate AI tools are being introduced into the market, as discussed below. A CAUTIONARY TALE OF POTENTIAL MALPRACTICE AND SANCTIONS FROM POOR USE OF CHATGPT Steven Schwartz, a practicing New York lawyer for 30 years, used ChatGPT to prepare a brief for federal court. Mr. Schwartz found cases with citations that sup ported his client’s rather unorth odox claim. At least six cases he cited in a brief as filed were halluci nations that did not exist, with ficti tious quotes and internal citations. When the brief was filed, the fact that ChatGPT could halluci nate cases (including fabricated quotes from the cases) was well known within the legal technology community, but certainly not all lawyers were aware of this. Opposing counsel filed a response brief calling out the bogus cases and moving for sanctions. Mr. Schwartz, dis regarding that the opposing counsel’s filing presented a huge red flag, went back to ChatGPT to confirm that the cited law was correct. “I apologize for the confusion earlier,” ChatGPT replied. “Upon double-checking, I found the case Varghese v. China Southern Airlines Co. Ltd., 925 F.3d 1339 (11th Cir. 2019), does indeed exist and can be found on legal research databases such as Westlaw and LexisNexis. I apol ogize for any inconvenience or confusion my earlier responses may have caused.” The lawyer filed a response reaffirming the opinions repre sented good law without locating or reading the actual opinions. Apparently, the lawyer’s only legal research tool was Fastcase,

This situation has prompted a few federal judges to issue standing orders requiring counsel to submit affidavits that they either did not use AI in preparation of the brief or, if they did, a human checked the AI’s work.

SO WHY HALLUCINATIONS? The large language model AIs certainly appear to understand your queries and provide logical responses. Professor Kenton Brice, director of the Donald E. Pray Law Library at the OU College of Law, had a helpful analogy at our OBA Solo & Small Firm Conference program on ChatGPT and AI. He said to think of the game Mad Libs. The AI does not understand the meaning of its communication with you. To use Professor Brice’s analogy, if the AI is completing the sentence “A cat is _,” there are many possible word choices to complete the sentence. A cat is a mammal. A cat is black. A cat is a feline. The AI chooses based on its ingestion of hundreds of millions of online pages and the context of the query or discussion. The sur prising thing is how often it selects the perfect word or phrase. The remarkable thing is not that it gets things wrong, but that it mostly gets things right. But since it doesn’t understand truth or falsity, it doesn’t apply those values, just probabilities. Of course, “mostly correct” is not an appropriate standard for lawyers when working for clients. When you use ChatGPT for drafting, consider its output a

receiving a list with useful infor mation you didn’t have in seconds. I have Google, DuckDuckGo and ChatGPT installed on my phone. I use Google and DuckDuckGo when I want an answer, a location or some other basic information. But if I want an explanation, ChatGPT is the first option for a search. I also note that OpenAI has provided a fix for the concern of “your friend” sharing informa tion about your queries. There is a setting to prevent your ChatGPT queries from being further used to train the system. Once lawyers start to do research for client mat ters, they will probably want to enable that setting – not because it’s likely information would be compromised, but just because we don’t understand everything, and it is the safer course of action. 1 Even with that safeguard enabled, lawyers will still want to use discretion formulating their queries and avoid unnecessarily using client information when a hypothetical will do. But exercis ing caution doesn’t suggest your search history is readily available to others.

70 | AUGUST 2023

THE OKLAHOMA BAR JOURNAL