Four Medical AI Decisions Show What Oversight Must Track

Four Medical AI Decisions Show What Oversight Must Track

Start with the decision

A system that screens a retinal image, alerts a stroke team, guides an ultrasound scan or flags a possible polyp may be called “medical AI.” But each does a different job, and each can affect care in a different way. Oversight should track the decision a tool influences and the consequences of an error, not just the broad label “AI.” (FDA)

The available sources cannot substantiate an account of the White House–company pact, Senator Thune’s proposal or the congressional debate over frontier AI rules. They do support a narrower question with direct consequences for patients and clinicians: what should oversight track when AI enters medical care? Four devices on the FDA’s public list illustrate distinct roles: screening, alerting, image acquisition and detection. Their regulatory entries identify intended uses and status, but do not, by themselves, establish how well the products perform in routine care. (FDA)

Artificial intelligence, or AI, is software that performs tasks such as recognizing patterns or generating predictions. In medicine, the task matters. Flagging a possible abnormality in an image is not the same as guiding image capture or advising a clinician on what to do next. A regulator’s product listing helps identify what a system is meant to do; it is not a report card on outcomes across hospitals, patient groups or later software versions. (FDA)

The FDA’s separate guidance on predetermined change control plans addresses one part of the product lifecycle: how a manufacturer can describe planned changes to an AI-enabled device and the process for managing them. Together, the device list and guidance point to three questions that should not be conflated: What is the product authorized to do? What evidence shows it works for that task? What happens when the software changes? (FDA guidance)

Five routes into clinical practice

The World Health Organization’s 2024 guidance on large multimodal models identifies five broad health applications: clinical care, patient-guided use, administrative work, medical education, and research and drug development. Large multimodal models are AI systems that can work with more than one kind of information, such as text and images. These are use categories, not interchangeable product types or stages of approval. A single system may serve more than one role, and the consequences of error depend on where it is used. (WHO)

Each category raises different questions. In clinical care, does the system improve or distort a health professional’s decision? For patient-guided use, could someone mistake an answer for a diagnosis or miss a reason to seek care? In administration, does generated or reorganized information preserve what matters in the record? In education, are learners being taught reliable material? In research and drug development, can the output be checked against the underlying evidence? These are prompts for matching safeguards to the task, not claims that every system fails in these ways.

The Stanford AI Index’s 2026 medicine chapter provides a broad view of AI in medicine, while the FDA list offers a product-level view of devices that have entered regulatory review. The distinction matters: a report about activity across a field and a regulator’s entry for a named device do not answer the same question. Neither, on its own, establishes clinical benefit. (Stanford HAI; FDA)

For patients and clinicians, the useful first question is concrete: what decision or task does this system influence? Knowing whether a product assists image capture, prioritizes a scan or offers a diagnostic result helps identify what evidence and human review matter most. “AI-powered” says much less.

Four devices, four different roles

The FDA’s list includes IDx-DR, a system intended to detect more than mild diabetic retinopathy from retinal images. The FDA listing places its authorization in 2018. Its assigned role is specific: screening, not general-purpose medical advice. The listing documents regulatory status and intended use, not how often the system changes treatment or improves outcomes in every setting. (FDA)

Another 2018 example is Viz.ai’s stroke-related software, designed to analyze medical images and alert care teams to a suspected large-vessel blockage. It is a notification and workflow tool, not a replacement for a clinician’s diagnosis. Its value depends partly on whether a timely alert helps a team act; risks include missed alerts or interruptions that do not help care. The FDA listing identifies the device and its intended function, but does not establish the effect of every alert on patient outcomes. (FDA)

Caption Guidance, listed by the FDA in 2020, uses AI to help guide the acquisition of cardiac ultrasound images. GI Genius, listed in 2021, is designed to identify possible polyps during colonoscopy. One assists image acquisition; the other flags an area for a professional to assess. Their different roles show why a single pass-or-fail label for “medical AI” would be too crude: a system’s place in the workflow shapes what a meaningful evaluation should measure. (FDA)

These four timelines, 2018, 2018, 2020 and 2021, are not a count of all medical AI or a measure of its clinical success. They are a small set of named examples from the FDA’s device list. Read carefully, they show AI entering care as a screening system, an alert, an imaging assistant or a detection aid. Look beyond the shared label: what is the system allowed to do, what must a person still decide, and what evidence supports its use? (FDA)

Authorization is not the finish line

A medical device’s first authorization answers a bounded question about its stated purpose and the evidence submitted for review. It does not mean the software can be changed without further scrutiny, or that the same performance will follow automatically in a new setting. That is one reason the FDA has issued guidance on predetermined change control plans for AI-enabled devices. (FDA guidance)

A predetermined change control plan, or PCCP, describes anticipated software changes and the procedures a manufacturer will use to develop, validate and implement them. In plain terms, it gives regulators and manufacturers a way to discuss some planned updates in advance rather than treating every change as an entirely new product. The FDA guidance concerns recommendations for such plans in marketing submissions; it is not blanket permission to alter a system without oversight. (FDA guidance)

This framework focuses attention on a central challenge in machine learning, a way for software to identify patterns in data and use them to make predictions or classifications. When software is designed to change, or is updated using new data, a reviewer must understand what changed and whether the change affects the product’s intended use or performance. The FDA’s guidance offers a regulatory route for describing those changes; it does not prove that every update will be harmless or beneficial. (FDA guidance)

For practitioners, version information belongs in the product’s safety story. A clinic evaluating a tool should know which version it uses, what task that version is intended to perform, and how the supplier handles material updates. A regulator’s review and a clinic’s continuing attention answer different questions; neither makes the other unnecessary.

Evidence should follow the tool

Evidence should match the job each device is meant to do. For IDx-DR, a useful evaluation would report how reliably the system identifies the specified level of diabetic retinopathy in the intended screening population, including missed cases and false alarms. For Viz.ai’s alerting software, relevant questions include whether alerts reach the care team in time to affect workflow, how often alerts are missed or unhelpful, and whether the claimed benefit is measured in patient care rather than notification speed alone. These are evaluation criteria, not findings established by the FDA listings. (FDA)

The same principle shapes evaluation of the other two products. For Caption Guidance, evidence should address whether the system helps users obtain images adequate for the intended examination, and whether that holds for the people and settings in which it is used. For GI Genius, an evaluation should distinguish detection performance from downstream benefit: identifying a possible polyp is not itself proof that the tool improves outcomes. Across all four, results should make clear who was studied, where the system was used, what clinicians did with its output, and which software version was tested.

A product listing is a starting point, not a scorecard. The FDA list identifies AI-enabled devices and their stated functions, but an entry alone does not establish performance in a particular hospital or patient population, or after a software change. Claims about benefit should be tied to study results that match the product’s intended use and setting, not inferred from regulatory status. (FDA)

The WHO’s guidance on large multimodal models also cautions against treating broad capability as proof of dependable use in health. A system that can produce fluent text or combine information types may still produce inaccurate or incomplete output. The WHO frames health applications across five areas, underscoring that risks and safeguards depend on where a system is used and who relies on it. (WHO)

The Stanford AI Index’s medicine chapter and recent medical research in journals such as Nature Medicine and PLOS Digital Health can help fill in the evidence landscape, but claims should match what each study actually tested. A result in a controlled evaluation may not establish the same outcome in routine practice. Since the supplied sources do not, on their own, support a detailed comparison of clinical performance across products, this article does not rank the four devices or claim they improve care by a specified amount. (Stanford HAI; Nature Medicine; PLOS Digital Health)

For a reader assessing a claim, three checks help: identify the exact product and version, find its intended use, and distinguish authorization from evidence of benefit in the setting where it will be used. This modest standard keeps a compelling demonstration from being mistaken for proof that a tool works equally well everywhere.

Governance should match the decision

The medical examples point to a wider principle for AI oversight: rules work best when tied to a system’s real function and the consequences of error. An image-acquisition assistant, a screening tool and an alerting system do not make the same decision. They should not be evaluated as if they do. The FDA’s device list and change-control guidance illustrate how product purpose and planned changes can shape review. (FDA; FDA guidance)

This does not resolve the separate federal debate described in the editorial brief. The validated sources here offer no evidence about Thune’s proposal, company commitments to the White House, or the scope of congressional authority. Treating clinical-device guidance as proof of how those political negotiations will unfold would blur two different questions. The medical sources support a narrower insight: a workable framework must account for both initial deployment and later change.

The FDA should continue to make its device listings clear about intended use and regulatory status. If Congress legislates on AI, it should require agencies to explain how responsibilities fit together rather than treating voluntary company pledges as a substitute for statutory authority. That recommendation follows from the limits visible in the sources: public inventories help people see what has been reviewed, but they do not establish ongoing performance or resolve who is accountable for updates. (FDA; FDA guidance)

Over the next year, the practical test for medical AI governance will be whether regulators, manufacturers and care providers can make product purpose, software changes and supporting evidence easier to trace. That is a forecast, not a confirmed policy timetable. For patients and practitioners, the next step is clear: ask what the tool does, what evidence supports its use, and what happens when the software changes.