A Small Practical Test: What Can an AI-Assisted Trial Comparison Leave Out?

Putting three trials into a table can take only a few fields. Deciding whether they inform the same development question requires more information. We used three completed lung-cancer studies for a public-material demonstration, examining the gap between fixed-field extraction and further reading.

Here is what was actually done.

On 27 September 2026, three records were saved through the ClinicalTrials.gov public API. A script first extracted identifier, disease label, phase, allocation, primary outcome names and record update date. AI-assisted reading then reviewed public eligibility and outcome descriptions against the original text. The first stage did not extract all eligibility clauses; that scope limitation was part of what we examined.

This was not an accuracy comparison of commercial products. There was no independent physician scoring, real patient information, enrollment assessment or time-saving measurement. It illustrates one small information-processing exercise and cannot establish a general model error rate. Source records, the first-stage field table and review notes were retained for the exercise.

The studies were KEYNOTE-189, CheckMate 9LA and IMpower150. The records list completed, randomized Phase III research and disease labels referring to non-small-cell lung cancer. Those labels alone make the rows look like closely comparable studies.

Eligibility text reveals a histology difference.

Public inclusion summaries for KEYNOTE-189 and IMpower150 concern nonsquamous disease, while CheckMate 9LA includes squamous and nonsquamous disease. The broad condition field in the first table did not convey that difference. It matters when discussing populations, treatment context or comparisons of results.

A test requirement is different from a positivity requirement.

The CheckMate 9LA criterion requires central-laboratory PD-L1 results without specifying a positive cutoff in that bullet. Converting “results required” into “positive patients required” changes the population description. Original wording keeps the interpretation within what the source states.

Exclusions may contain consequential exceptions.

The same record excludes untreated CNS metastases but describes conditions for adequately treated disease, neurological recovery to the stated baseline and timing. Reducing it to “brain metastases excluded” loses that exception. Review the clause rather than substituting a disease label for all requirements.

Do not delete analysis populations from outcome names.

The IMpower150 record specifies populations such as Teff-high WT and ITT-WT for some outcomes. The first extraction preserved full names, but their qualifications still require interpretation. Shortening the cell to “PFS, OS” would remove that relationship. The public eligibility summary also cannot supply every related population rule.

The records’ latest posted updates fall in 2024, 2025 and 2021. A common retrieval date is therefore not a common level of document currency. Registry information is not a full protocol or necessarily every amendment. These are checkable public-material differences rather than a definitive comparison of full protocols.

Pair a summary with a review table.

The summary provides orientation. The review records histology, treatment context, testing and cutoffs, exceptions, analysis populations and gaps. Preserve sources for individual statements. Treat missing information as unknown rather than permission or prohibition.

This test does not establish treatment superiority or quantify business value. It shows a specific limitation: fixed fields help organization but cannot independently support a competitor-trial judgment. Retaining clauses, analysis populations and uncertainty makes AI-assisted output more suitable for development discussions.