NCSB Journal Summer 2026

authority. The second is AI embedded with in established legal research platforms, such as Westlaw, Lexis, or Bloomberg, which operate within controlled databases and incorporate citation-checking and verifica tion safeguards—though, as the experiences described below demonstrate, even these tools are not infallible. The third category consists of firm-internal or document review AI tools, often used to summarize discovery, organize records, or analyze large datasets, where confidentiality and supervi sion concerns predominate. Each category presents different benefits and different risks. Treating them as functionally equiva lent invites both ethical missteps and false confidence, particularly when lawyers assume that the presence of a familiar plat form name eliminates the need for inde pendent verification. AI accumulates an almost immeasurable database of books, articles, websites, and conversations, and from this, it learns pat terns of words, facts, and common reason ing. When asked a query, it applies statisti cal analysis and relationships to formulate a response. The problem is that the non-legal databases are not infallible, and AI often makes mistakes. It can provide the attorney with a poignant quotation for a legal issue and provide a citation from which it was derived. But the quotation and citation may be entirely “make-believe.” When this occurs, it is appropriately called a “halluci nation.” AI Hallucinations: Fabricated Authority with Convincing Style One of the most troubling aspects of gen erative AI is its tendency to produce “hallu cinations”—confident, well-structured responses that contain fabricated facts, quo tations, or citations. In legal contexts, hallu cinations often take the form of: • Nonexistent cases • Real cases cited for propositions they never addressed • Quotations that are eloquent, persua sive—and completely fictional. AI does not lie intentionally—but it con fidently generates fiction that looks like law. It generates content that statistically resem bles legal writing. Some people wonder why generative AI hallucinates, and the answer is that it wants to make the user happy. It cre ates content based on its evaluation of the user’s request and its prediction of the best

answer. That "best answer" isn’t always about accuracy; it’s more about coherence and patterns that will keep you engaged. As it learns more, it will become more accurate, but it’s still got a long way to go. Unfortunately, plausibility is not accuracy. 3 When attorneys fail to verify AI-generated output, they risk submitting fiction to the court. 4 The consequences can be severe. Here is what happened to us. Fred’s Story: I represented a client in fed eral court. I prepared a summary judgment brief and consulted with two well-respected attorneys who were experts regarding the legal issues involved. They volunteered to review my brief and make suggestions on its content. The attorneys then collectively sent me several paragraphs of compelling argu ments, complete with persuasive quotations and citations. I gladly incorporated them into my brief. There is no worse feeling than receiving a copy of the defendant’s brief, where the defendant informed the court that the quo tations and citations I had incorporated were hallucinations. I had signed a certification that I did not use AI to write the brief, which was true, but I failed to verify the authentic ity of the supplemental authority I was given. This violated my certification. The fault does not lie with the consulting attorneys because my duty to verify the authorities is non-del egable. My penance was to resubmit a cor rected brief and to warn the Bar of the con sequences of non-approved AI sources. The moral is unavoidable: delegation does not eliminate responsibility. Rob’s Story: I am co-counsel with Fred on the same case in federal court. We agreed to take over the case with very little time left for discovery. We had to review a large vol ume of discovery documents and become deeply familiar with the case and regulatory law in what was, for me, a new area of law. I felt I was behind on technology and hired a consultant to help me get up to date with AI. While writing briefs on dispositive motions, I relied on a commercial legal research pro gram’s rendition of several quotes supporting the points I made in the brief. The cases I cited were correct, but the quotes were hallu cinated. I checked the cases but not the quotes. Again, I was the person who certified that I had authenticated the cited informa tion. Before this incident, I was very confi dent I was using legal AI as effectively as pos sible, only to be shown I was wrong.

Fortunately, the consequences of my mistake have been rectified. Hopefully, my experi ence can help others avoid the same embar rassment, anxiety, and frustration. The Temptation of Over-Reliance on AI Make no mistake about it, AI will forever change how legal drafting and research are conducted. Those attorneys who do not become proficient in the legitimate use of AI will be disadvantaged by opposing counsel who use AI. At the same time, unchecked enthusiasm presents an equal risk. AI’s effi ciency creates a powerful temptation: faster briefs, smoother prose, instant citations. But speed can come at the expense of under standing. When lawyers rely too heavily on AI, they risk substituting output for analysis. In addition to the hallucination issues, AI creates other dangers for the practitioner. Erosion of Professional Judgment Regardless of how efficient AI is and will become, professional judgment cannot be automated. Relying on AI to solve a client’s legal issue is not only a disservice to the client, but it is the path to legal malpractice. Legal judgment is not just correctness; it is also context, nuance, and strategy. AI does not know your judge, jurisdiction quirks, the human dynamics of opposing counsel, or the client’s risk tolerance. Professional judgment develops through reading cases, uncovering weak arguments, and reconciling conflicting authorities. This legal skill will atrophy if there is an over-dependence on AI. Use of AI to draft pleadings or briefs may seem to pro duce polished work quickly. But this is an illusion of mastery and can mask gaps in understanding of the issues and the law. Think of AI as a junior assistant or a second set of eyes, but not as a decision-maker or a proxy for ethical responsibility. Ethical Violations Our State Bar has already adopted formal ethical guidance regarding the use of artificial intelligence in legal practice. 2024 FEO 1 addresses key issues, including when AI assisted legal research is appropriate, whether confidential client information may be uploaded to AI platforms, whether attorneys must disclose their use of AI to clients, the potential impact of AI on hourly billing practices, and the lawyer’s duty to independ ently verify AI-generated research. As the use of AI continues to expand, further updates

9

THE NORTH CAROLINA STATE BAR JOURNAL