Patents

In recent months, we have been working extensively with customized AI tools to assist with both administrative and substantive aspects of our practice. Tools like these can deliver real efficiency gains, and in some cases, they let us operate at a level of depth that was not previously practical. They are, in our view, here to stay – the question is no longer whether to use AI but how to use it responsibly.

That said, clients should not assume that these are turnkey solutions where a push of a button produces a finished product. Using these tools, at least for now, requires close human scrutiny and supervision. Most of the public discussion of AI’s limitations focuses on so-called “hallucinated” results – fabricated cases, made-up citations, invented quotations. Those are real problems, but in our experience they are not the most important ones. The bigger problem is more subtle, and it stems from a feature of AI output that initially looks like a strength: polish.

Polish Can Mask Substantive Flaws

AI work product typically arrives in a formal, well-organized form. Headings are in place, citations are formatted, conclusions are stated with confidence. That polish creates an air of credibility that can mislead a reader into believing the underlying analysis is as solid as the presentation. However, this finished-looking product is the starting point with AI because from there, we have to scrutinize the analysis in detail, including looking for the unstated inferences, logical leaps, and just plain oversights the AI models make.

As an example, we recently used AI to carry out a right-to-use analysis of the independent claims of over 30 patents versus seven design options. We got a finished report that grouped the design features that presented the greatest infringement threat and ranked them by color-coded threat levels. It was an impressive feat. However, upon closer scrutiny it was clear that in certain cases the claims were misapplied and features of the designs were not properly accounted for. It took 13 iterations to get to the finish line. AI saved us time, and the mode of analysis (ranking design features by infringement threat levels as opposed to merely identifying the references with the “closest” claims) was one we had been unable to do manually in many cases. So, it was absolutely worthwhile, but that was true only because of the level of human scrutiny that was applied.

Case Law: Language That Sounds Relevant

In case law analysis, we have observed a related problem. AI tools often cite cases for propositions they do not actually support because some language in the case sounds relevant in the abstract. The factual posture of the cited case may differ in ways that make the quoted language inapplicable, but the AI does not flag the distinction. This is not limited to general-purpose tools. The LEXIS Protege platform, which uses a “closed universe” of LEXIS legal source materials precisely to reduce hallucinations, still produces case citations that do not stand up on closer reading. The cases exist; the quoted language exists; what is missing is the analytical step of asking whether the cited authority actually supports the proposition being asserted.

What This Means for Clients

We understand and appreciate why there is a strong push from clients to ensure their outside counsel are leveraging AI to reduce their legal spend. However, it is important to understand both the benefits and limitations of AI to set realistic expectations. Useful AI work product, in our experience, is the end result of an extended “dialogue” with the AI model. Working with AI on substantive issues is at times like cross-examining an expert witness.

We will continue to refine the AI tools that have become part of our daily workflow, and we expect to talk about them more in coming months. For now, our message to clients is the same one we have given ourselves: the work product is impressive, but only with the appropriate level of human involvement.