I have had at least five people ask me if I was going to blog about Kim Kardashian failing the California bar exam and blaming it on getting bad advice from ChatGPT. I have tried to resist, but here we go.
First off, I have no idea how you study for the bar exam with ChatGPT. There are a bunch of bar exam prep classes that walk you through the test and help you prepare. If she did not do that then that is her fault and likely the reason she failed it in my opinion. That and not actually going to law school.
If she did rely on ChatGPT or other AI programs for legal advice, she is certainly not alone. Every week I get calls from people who are trying to confirm what AI told them about their legal situation. And like Kim K, they usually discover that ChatGPT does not know what it is doing. It is like taking legal advice from your washing machine.
AI programs can not deal with nuance which is something almost every legal situation has. They are regurgitating sentences based on programming and info they have been given. That is not real legal advice. And if it is wrong as often as it is right, that is a huge problem. It would be a huge problem if it was only wrong 1-2% of the time. You would not want a lawyer who you could never be sure if they are correct.
A recent example was an injured worker who insisted to us that his case was worth $50,000 because AI told him it was. In his case, he did not go to a doctor until 60 days after the injury and also did not report to the first medical provider that he was hurt at work. That greatly affects his case. We asked him what AI said about those factors and of course there was no answer to that. Nuance and unique case facts are everything when it comes to offering a real legal analysis.
In another case, a client had used ChatGPT to come up with case law. The problem was that he found criminal case law and he was involved in a civil law suit. In other words, if those cases are actual real cases and not hallucinated by AI, they are still not relevant in any way.
I get why people want to figure out things on their own or hope to save some money. But the truth is that ChatGPT is not it. It has no idea what it is doing and even if you ask the questions perfectly, it can not anticipate case issues, responses by the other party, how the Judge will feel, variables that can change the correct answer and many other things that are relevant.
I had one attorney tell me about a client who did not want to pay $500 to translate a legal document, but instead wanted to rely on a ChatGPT language translation. If that was entered into court and there were any mistakes, the client and the lawyer would likely face a significant fine.
In another case, we got called the day before the statute of limitations to file a lawsuit would cause the case to be barred forever. When I asked the injured person why they waited so long to reach out for legal help, they told me that ChatGPT was acting like a paralegal advisor for them and they felt comfortable handling the case on their own.
When I asked if the AI gave advice on medical liens, they told me that they did not know anything about that. When I asked if the AI advised them about the time limits to sue, they told me it did not. When I asked if the AI talked to them about UIM coverage or verifying policy limits, he said it did not. I ended up not being able to help because they came to me with too little time to spare. End result is they lost their chance to recover anything.
ChatGPT is limited to what it knows and what you ask. It can spit out generalities, but you will never know what you or it is not thinking of and how that can harm you. And you will never know when it is giving you advice that does not apply to your situation or is just plain wrong.
Back to Kim K. Her not passing the bar exam will not change her life in any meaningful way. I don’t think anyone anticipated she was ever going to actually work as a lawyer and she clearly does not appear willing to do the hard work needed to accomplish her goal of a law license. I can not even fathom how you would “study” for the bar exam by asking a computer questions. If you are a law student, I can not recommend enough that you avoid this route. Take the normal bar exam prep classes that have an actual track record of helping people pass the exam.
And if you are a regular person with legal issues, while there is nothing wrong with researching how cases work or what might happen, if you are making decisions off of AI that is likely at least partially wrong, you are risking your whole case falling apart.
