Introduction

Artificial Intelligence (AI) has made things easier, faster and more efficient. A major concern regarding AI is its reliability and credibility. Artificial intelligence can also produce information that appears to be true and credible but is in fact false or nonexistent. With the increasing use of AI in legal research and decision-making, there is a growing risk of AI generating fabricated case law, incorrect citations, or non-existent judicial precedents. Such erroneous legal information presented as fact is commonly referred to as a “legal hallucination.”

AI Hallucination in Litigation

The growing trend of using AI can also be observed in the legal field. In the current scenario, lawyers and legal counsel increasingly use AI for legal research and other professional purposes. The rapid integration of Large Language Models (LLMs) and generative artificial intelligence into legal practice has transformed legal research, document drafting, and workflow automation. However, this technological advancement has also introduced a significant vulnerability known as legal hallucination.

Generative artificial intelligence systems are fundamentally based on probabilistic next token prediction, not on deterministic legal reasoning or factual verification. So, when prompted to search for judicial precedents supporting a certain proposition, these systems may prioritise linguistic fluency and relevance over factual accuracy. This can lead to the confident production of fabricated case law, incorrect citations, fictional judge names, made up legal principles and non-existent paragraph extracts. A legal system founded on the doctrine of stare decisis, where the foundation of legal authority lies in judicial precedent, the creation of non-existent case law represents a grave danger to the probity of the justice delivery system.

Supreme Court Recent Judgment

On 2 July 2026, the Supreme Court delivered what is thought to be its first significant verdict on the thorny issue of AI-hallucinated case law. In a landmark finding, a Bench of Justices Pamidighantam Sri Narasimha and Alok Aradhe overturned orders of the National Company Law Tribunal (NCLT) and National Company Law Appellate Tribunal (NCLAT), which had drawn upon judicial precedents that, according to the Supreme Court, had not in fact been decided by any court. The ruling is notable for including a number of important observations on the use of artificial intelligence in Indian Courts.

Background of the Case

That the case before the NCLT and NCLAT was filed under Section 7 of the Insolvency and Bankruptcy Code, and it was appealed in Supreme Court Civil Appeal No. 11950 of 2025, which led to the current issue. During the arguments before the Supreme Court of India, the counsel for the appellant pointed out several irregularities in the impugned order of the NCLT dated 28.8.2024, which were upheld by the NCLAT in an order dated 11.9.2025. Upon scrutiny, the Court noted two types of irregularities. First, certain “precedents” cited by the NCLT did not appear to exist in any law reports. Second, in certain instances, the NCLT had relied upon judgments that did contain the quoted paragraphs, but which had not actually been reported in any law reports.

According to the affidavits filed before the Supreme Court by J&K Bank, the “precedents” cited in the NCLT orders had not been placed on record by either party. The NCLT itself appears to have conducted independent research using an AI tool to generate “relevant” precedents, and the NCLAT had affirmed the order of the NCLT without detecting any fraud.

Supreme Court Held

The Supreme Court set aside the orders of the NCLT and the NCLAT and issued several important observations. First and foremost, it held that ‘a judicial or quasi-judicial decision, which is, in whole or in part, based on falsified (hallucinated) material, is, in law, a subversion of the rule of law.’ Crucially, this applies even if the falsified information had no nexus to the issue actually before the court.

The Court pointed out that decisions must be vacated even in cases where only an “iota” of the information relied upon to decide the case was fabricated or hallucinated by the AI, because such reliance on falsified information, however “insubstantial,” was a “contamination of the adjudicative process.” The Supreme Court further held that a policy of zero tolerance must be adopted by both the Bar and the Bench when it comes to reliance on hallucinated information generated by AI.

For lawyers, this means that using AI to identify, extract, or apply precedents without verifying their authenticity and relevance to the issues actually before the court constitutes professional misconduct. For members of the Judiciary, it means that using unverified AI-generated information when drafting orders or judgments, even when done “in good faith,” is also a breach of judicial ethics.

The Court further held that it makes no difference whether the hallucinated information was inserted by counsel or by the court itself – in either case, the decision of the court must be set aside. The Supreme Court directed the Bar Council of India to constitute a committee to examine the various issues raised in the case and report its recommendations to the Court.

Clarifying The Limits of Zero Tolerance: The Supreme Court Warns Against Blaming AI Itself

It is important to note that the Supreme Court did not condemn the use of AI per se. The Court specifically observed that the use of AI for conducting research and even for preparing draft judgments or orders is not prohibited, provided that no unverified information is placed on record or presented as genuine. This distinction is crucial. The Court clarified that what is prohibited is the use of AI-generated and unverified information as if it were genuine. It further warned that the failure to distinguish between verified information and unverified information presented as authentic could vitiate any order or judgment passed by a court or tribunal based on such erroneous information.

Conclusion

The judgment in Pooja Ramesh Singh v. J&K Bank is a significant step toward responsible use of artificial intelligence in the legal profession. The use of artificial intelligence in the legal field has been outlawed by India’s Supreme Court. However, clearly mentioned lawyers and judges should personally check the credibility of the information. As AI tools become more and more embedded in the courts, tribunals and law firms, the verification of precedents, statutory provisions and other legal information must become a core professional responsibility. Properly implemented, AI should be a tool to assist legal professionals, not a replacement for their judgment, diligence and accountability. The efficiencies offered by technology should never come at the expense of accuracy or the integrity of the administration of justice.

 

Author: Sachin Sharma, Associate
Co-Author: Nitisha Rai and Shaurya Darsh, Intern