Translating Super Intelligence to Clinical Practice: Methodological Challenges and Safety Limits in Autonomous Healthcare

Abstract

Background: Public and professional discourse increasingly describes medical artificial intelligence (AI) in the vocabulary of “superintelligence”, implying that clinical autonomy is a matter of engineering time rather than of evidence. This framing is rarely tested against the methodological quality of the studies on which it rests.

Objective: To appraise, in a deliberately sceptical manner, the methodological obstacles and the safety limits that separate benchmark-level machine performance from accountable, autonomous clinical care.

Methods: A structured narrative synthesis was conducted, reported in line with the principles of PRISMA 2020 and its scoping-review extension (PRISMA-ScR). Two evidence streams were combined: a supplied bibliographic corpus of 117 records, which was screened to 41 topically eligible items, and 47 purposively selected external sources comprising systematic reviews, trials, reporting standards, regulatory instruments and safety-science literature. Findings were organised into evidence quality, validity, human-factor, equity, governance and liability domains.

Results: Across the benchmark literature, headline accuracy was repeatedly accompanied by retrospective designs, limited external validation, high risk of bias, scarce prospective evaluation and weak reporting. Documented deployments revealed failures that laboratory metrics did not predict, including poor sensitivity of a widely implemented sepsis alert, proxy-driven racial bias in a population-health algorithm and workflow friction in a field-deployed retinal screening tool. Human oversight was found to be a fragile safeguard because of automation bias and alert fatigue. Only a narrow class of tightly bounded tasks currently has trial-grade evidence supporting autonomous operation.

Conclusions: Autonomy in healthcare should be earned incrementally, task by task, through prospective, externally validated, equity-audited and legally accountable evaluation. A five-level Clinical Autonomy Ladder and an evidence-gated promotion protocol are proposed. Claims of superintelligent medicine are, at present, aspirations about the future and not findings about the present.