AI Hallucinations in Legal Research: Supreme Court’s Zero-Tolerance Judgment in Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. (2026)

Context

Artificial Intelligence (AI) has emerged as a transformational technological advancement of the twenty-first century, revolutionising sectors such as healthcare, finance, education, and government. The legal industry has increasingly incorporated AI-driven technologies that aid attorneys in legal research, document review, contract drafting, predictive analytics, and case management. Although these advances have markedly enhanced efficiency and accessibility in the justice delivery system, they have concurrently produced intricate legal, ethical, and procedural issues. One of the most significant issues is the occurrence of AI hallucinations, in which AI systems produce false information, such as non-existent court precedents, legislative sections, or legal principles, and assert them as legitimate legal authority.

The risks associated with AI-generated hallucinations gained extraordinary importance in India when the Supreme Court, in Pooja Ramesh Singh v. Jammu & Kashmir Bank Ltd. &Anr., 2026 LiveLaw (SC) 653, issued a pivotal decision denouncing the utilisation of counterfeit AI-generated judicial precedents in legal proceedings. The Court implemented a definitive “zero-tolerance” policy regarding fraudulent AI-generated citations, asserting that judicial rulings based on fabricated precedents constitute “no decision in the eyes of the law.” Consequently, the Court annulled the judgements of the National Company Law Tribunal (NCLT) and the National Company Law Appellate Tribunal (NCLAT), and established a significant jurisprudential framework for the responsible application of AI in the legal field.

This judgement signifies a pivotal point in Indian legal history as it both embraces technological advancement and reaffirms that judicial decision-making must remain entirely under human oversight. It emphasises that while AI can serve as a significant research aid, it can never supplant judicial thinking, legal analysis, or professional accountability. Moreover, it strengthens the constitutional principles of justice, openness, accountability, and the rule of law by mandating that every judicial judgement be based on genuine legal authority rather than algorithmic creation.

Read Also: Judiciary Current Affairs

The Impact of Artificial Intelligence on Legal Practice

Artificial Intelligence has significantly transformed the way legal practitioners do their business. Contemporary AI systems can swiftly evaluate vast quantities of judicial judgements, discern pertinent statute sections, provide legal summaries, compose pleadings, and even forecast litigation results based on previous judicial patterns. These capabilities have significantly decreased the time historically necessary for legal research while enhancing access to legal information for practitioners, litigants, and judges.

Worldwide, courts have begun trials of AI-assisted technology for administrative tasks like case scheduling, document management, translation of judicial records, and initial legal research. India has similarly adopted technological reforms via initiatives like the e-Courts Project, digitisation of judicial records, virtual hearings, electronic filing, and AI-driven translation tools such as SUPACE (Supreme Court Portal for Assistance in Court Efficiency) designed to aid judges in case preparation.

Nonetheless, AI is inherently probabilistic rather than truly intelligent. Large Language Models provide replies by forecasting statistically likely word sequences rather than independently validating legal authority. As a result, these systems may reliably generate completely fictitious court precedents, erroneous statute sections, or misrepresented legal doctrines- a phenomenon referred to as AI hallucination.

In most business applications, such mistakes may only cause discomfort to consumers. In a judicial system, however, counterfeit legal authorities jeopardise the validity of adjudication, as courts obtain their authority from precedent, legislative interpretation, and constitutional reasoning. The integrity of the legal process necessitates complete legitimacy in every reference utilised throughout judicial decision-making.

Context of the Present Case

The Supreme Court litigation arose from insolvency proceedings commenced by Jammu and Kashmir Bank Ltd. under Section 7 of the Insolvency and Bankruptcy Code, 2016 against Essel Infraprojects Ltd., which had provided a corporate guarantee for financial facilities granted to Pan India Utilities Distribution Company Ltd. The National Company Law Tribunal in Mumbai accepted the insolvency application on 28 August 2024, noting a financial shortfall of almost ₹87.43 crore. Thereafter, the National Company Law Appellate Tribunal ratified the admission ruling on 11 September 2025.

Before the Supreme Court, Senior Advocate Madhavi Divan, representing the suspended director of the corporate debtor, contested both rulings on an exceptional basis. She illustrated that the tribunals had depended on various legal authorities that either were nonexistent or had been misattributed to substantiate claims for which they held no jurisdiction. The alleged precedents referenced in the contested orders included:

• State Bank of India v. Shree Ram Urban Infrastructure, 2020 SCC Online Supreme Court 341
• Everest Kento Cylinders v. Union of India, (2015) 2 SCC 1
• ICICI Bank v. Urban Infrastructure Real Estate, (2019) 16 SCC 528

An affidavit submitted to the Court verified that these authorities were either wholly fake or untraceable in any accepted legal database. The respondent further explained that these judgments had not even been referenced by counsel during arguments but had instead been independently relied upon by the NCLT. This alarming discovery revealed that AI-generated fabricated precedents have infiltrated judicial thinking, forcing the Supreme Court to investigate the broader institutional ramifications of artificial intelligence in adjudication.

The Supreme Court’s Zero-Tolerance Policy

The Supreme Court acknowledged the seriousness of the situation and took one of the most robust judicial stances worldwide concerning AI-generated legal hallucinations. The Court clearly said that a policy of zero tolerance must be upheld regarding false AI-generated legal precedents. The Bench believes that an advocate engages in professional misconduct by citing AI-generated judgements without properly checking their legitimacy. Concurrently, a judge perpetrates a similarly grave institutional error by depending on such contrived evidence when adjudicating cases.

The Court asserted that judicial rulings based on fraudulent precedents are entirely invalid as legitimate decisions. Such judgements erode public trust in the legal system and so warrant dismissal, regardless of whether the falsified authority significantly impacted the outcome. The Court noted that even a little instance of false precedent in the judicial reasoning process undermines the integrity of adjudication. As a result, both the NCLT and NCLAT decisions were annulled, and the case was referred back for further adjudication, wholly unaffected by fictitious authority. This rationale demonstrates an unwavering dedication to preserving the integrity, validity, and credibility of judicial decision-making.

AI Hallucinations: A Dire Threat to Judicial Integrity

Perhaps the most remarkable part of the decision is the Court’s vivid parallel connecting AI hallucinations with the emission of methyl isocyanide, the poisonous chemical responsible for the Bhopal chemical Tragedy. The Court acknowledged that a distinguishing feature of contemporary AI systems is their propensity to assertively produce false information while seeming very credible.

While scientists and engineers may persist in enhancing AI models to mitigate hallucinations, the Court underscored that the legal system cannot defer to technical perfection. From an adjudicative standpoint, falsified precedents function covertly until identified, by which point they may have already impacted court results. The juxtaposition with methyl isocyanide highlights that AI hallucinations exhibit three hazardous characteristics.

Firstly, they are imperceptible as constructed precedents frequently mimic authentic judicial quotations.

Secondly, they are pernicious since they permeate legal thinking without prompt recognition.

Finally, they are disastrous as they jeopardise the credibility of court decisions and erode public trust in the justice system.

The Court so determined that AI hallucinations are intolerable in adjudication under any circumstances.

Human Oversight of Artificial Intelligence

The Supreme Court did not entirely dismiss artificial intelligence. It embraced a balanced perspective, acknowledging AI as a significant technical asset but asserting that human thinking must prevail. The Court explained that artificial intelligence may aid judges in information organization, preliminary legal study, precedent identification, document translation, and administrative efficiency enhancement. Nonetheless, every phase of adjudication- from legal analysis to evidence evaluation and ultimate court reasoning- must be subject to comprehensive human oversight.

The Bench firmly said that adjudication cannot be assigned to machines, as judicial decision-making encompasses constitutional morality, human empathy, contextual reasoning, and value-based interpretation- attributes that exceed the capability of contemporary AI systems. Therefore, AI ought to function as an auxiliary tool rather than a decision-making authority.

Ethical Obligations of Advocates

The decision considerably broadens the ethical obligations of legal professionals in the digital era. Advocates have consistently had an obligation of candour to the court. This obligation now unequivocally entails the independent verification of every judicial precedent produced using AI-assisted research prior to its use in arguments or written submissions. The Court noted that lawyers cannot shirk professional accountability by ascribing errors to technical instruments. The advocate retains professional responsibility for AI-generated content after it is submitted to the court.

The reference of manufactured precedents without verification is professional misconduct, undermining the integrity of litigation and the administration of justice. Acknowledging the increasing significance of this matter, the Supreme Court instructed the Bar Council of India to establish an expert committee tasked with developing comprehensive guidelines for the responsible utilisation of artificial intelligence by legal practitioners, including suitable disciplinary measures for infractions.

Advancing Responsible Artificial Intelligence within the Indian Judicial Framework

The Supreme Court’s decision in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. (2026 LiveLaw (SC) 653) should not be understood just as a response to a singular mistake about falsified court citations. Instead, it signifies the commencement of a more extensive constitutional discourse concerning the role of artificial intelligence inside judicial institutions. As AI becomes more integrated into legal research, document writing, predictive analytics, contract management, and judicial administration, the legal profession faces a pivotal juncture. The issue is no longer whether artificial intelligence ought to be employed in litigation, but how it may be utilised responsibly without undermining the integrity, independence, and legitimacy of the judicial process.

The judiciary of India has a historical propensity for embracing technical innovation. The use of e-Courts, virtual hearings during the COVID-19 pandemic, digitisation of judicial documents, electronic filing systems, online case administration, and translation software has markedly improved access to justice. Artificial intelligence signifies the subsequent phase in this technological progression. In contrast to earlier technical tools, generative AI can generate original material, construct legal arguments, summarise judgements, and even create fictitious authority with notable language proficiency. This capacity requires a fundamentally distinct regulatory response.

The Supreme Court recognised this distinction by elucidating that AI itself is not the issue. The Court explicitly said that its decision should not negatively affect the lawful application of artificial intelligence in legal practice. Its concern was limited to the portrayal of fabricated or hallucinated content as authentic court precedent. Consequently, the judgment delineates a significant difference between AI as an auxiliary technical tool and AI as an unvalidated replacement for legal thinking.

The Human-in-the-Loop Principle

The judgment’s most notable jurisprudential addition is its clear acceptance of the “human-in-the-loop” premise. The Court asserted that adjudication must remain under the “total and absolute control” of human judges throughout all phases of the legal process. Artificial intelligence can aid courts in arranging material, summarising pleadings, locating pertinent precedents, or executing administrative tasks, but, the ultimate judgement of legal rights and responsibilities must remain solely a human responsibility.

This idea embodies globally recognised criteria for the ethical governance of artificial intelligence. Regulators across countries have repeatedly asserted that AI should enhance, rather than supplant, human decision-making in issues of fundamental rights. Judicial reasoning encompasses constitutional principles, ethical considerations, contextual analysis, and the reconciliation of conflicting interests, functions that cannot currently be emulated by algorithms.

The Supreme Court’s emphasis on human oversight thereby upholds a core constitutional principle: justice cannot be mechanised. Judicial independence stems not just from institutional autonomy but also from judges’ ability to use independent human judgment after meticulously assessing facts, law, precedent, and constitutional principles.

Comparative Perspectives: International Regulation of AI in Legal Proceedings

The issues highlighted by the Supreme Court are not exclusive to India. Courts in other countries have lately faced analogous issues stemming from generative artificial intelligence. In the United States, federal courts have penalised advocates for presenting falsified AI-generated authority in legal documents. The notable case of Mata v. Avianca, Inc. (2023) illustrated how attorneys utilising ChatGPT referenced wholly fabricated legal precedents. The court levied punishment upon discovering that the authorities were non-existent. Courts in Canada, Australia, Singapore, and the United Kingdom have mandated that advocates independently check any AI-assisted legal research prior to its presentation in judicial venues. The European Union’s AI Act employs a risk-based regulatory framework that prioritises openness, human supervision, accountability, and explainability in high-risk AI systems.

Conclusion

The Supreme Court’s decision in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. signifies a pivotal advancement in the developing law related to artificial intelligence and the administration of justice in India. The Court dismissed the NCLT and NCLAT judgements based on fabricated judicial precedents and outlined overarching constitutional principles on judicial integrity, professional ethics, institutional accountability, and responsible technological innovation.

The Court reiterated that criminal or civil adjudication cannot be based on counterfeit legal powers produced by artificial intelligence. It said that advocates engage in professional misconduct by using unverified AI-generated precedents, whilst judges exhibit a significant institutional failure by depending on such sources. The Court implemented a zero-tolerance policy regarding fabricated citations, asserting that any judgement tainted by false precedents is deemed “no decision in the eyes of the law.

The Court’s acknowledgement that artificial intelligence is neither intrinsically advantageous nor intrinsically perilous is equally important. Its validity is contingent solely upon prudent human oversight. By advocating for “human-in-the-loop” adjudication, instructing the Bar Council of India to establish ethical guidelines, and emphasising the essential function of independent judicial reasoning, the Court has established the constitutional basis for the responsible incorporation of AI into India’s justice delivery system.

The ongoing transformation of legal practice by artificial intelligence will not hinge on substituting judges with algorithms, but rather on maintaining that technology is consistently directed by the rule of law, professional integrity, procedural fairness, and constitutional morality, which are essential for the resilience of constitutional democracies. The future of justice depends not solely on artificial intelligence, but on the principled synergy between technical advancement and human insight- a balance that the Supreme Court has ardently endeavoured to maintain.

Sharing is caring!