Why Medical Record Review is So Hard — for LLMs and Humans

Medical evidence is core in most mass tort litigations. Determining whether a claimant used a product, experienced an injury, or had other medical issues or risk factors is critical throughout the litigation. Law firms rely on medical evidence to assess the viability of a case – an evaluation that evolves over time based on the changing litigation – and ultimately to understand potential case valuation outcomes. But the process of reviewing medical evidence is anything but straightforward. 

At Pattern Data, we’ve seen how technical barriers, linguistic nuances, and contextual demands make this process difficult for both humans and AI. Below, I’ll break down the key problems, explain how we address them individually, and show how our holistic approach transforms the review process.

The Challenges with Medical Record Review

1. Text: Most medical records are not “born digital.” Instead, they’re scanned documents or images that can’t be directly searched or processed.

2. Consistency / Specificity: Medical terminology is inconsistent. For example, “hepatocellular carcinoma” could appear as “HCC,” “hepatoma,” or “liver cancer.”

3. Ambiguity: Some terms are inherently ambiguous. For instance, “liver cancer” might refer to hepatocellular carcinoma or cholangiocarcinoma.

4. OCR Errors: OCR is not perfect. Words like “sarcoidosis” may be misread as “sarcodosis.”

5. Negations and Uncertainty: Phrases like “no history of cancer” or “not suspected” require precise interpretation to avoid misclassification.

6. Context: The meaning of terms depends on the sections or clinical note types in which they appear – a condition in a“Family History” section is interpreted differently than in a “History of Present Illness” section. Similarly, conditions within Pathology Reports can have vastly different clinical significance than those same conditions within other record types.

7. Visual Information: Critical information, such as audiograms, handwritten checkboxes on medical intake forms, medical device photos, implant stickers, and more, are often visual. Text-only approaches cannot capture the different meanings or interpretations of this information.

8. Special Cases: Litigation often requires handling unique scenarios, like extracting serial numbers from CPAP device photos for proof of usage. 

9. Deep Domain Knowledge: Medical expertise is required to understand nuanced terms like “Hürthle cell cancer” (thyroid cancer.)

10. Incomplete Records: Missing documents or insufficient information complicates the review. For example, you may not have the original diagnosis record for certain cancers without a pathology report. 

11. Incorrect Records: Sometimes, law firms upload the wrong records for claimants. 

12. Garbage: Unreadable scans or irrelevant data are common.

13. Evolving Requirements: Case priorities and evidence criteria change frequently over time.

14. An Infinite Number of Other Challenges: Medical review involves countless edge cases, like hierarchical data quality and cross-document inference.

A Holistic Approach to Solving These Problems

At Pattern Data, we address the complexity of medical record review through a unified strategy that combines advanced AI with human expertise. Our process begins by digitizing records with OCR and correcting errors with fuzzy matching. Fine-tuned models interpret medical terminology, disambiguate terms, and handle negations accurately. AI also segments records into meaningful sections to provide context for analysis.

We maintain a stable of machine learning, NLP, and computer vision models that analyze text, images, and document layouts to provide a contextual understanding of record types. This enables more accurate classification and extraction of relevant information.

Some litigations require specialized models to find and extract relevant information. We train and deploy custom models, using medical experts to help validate and refine outputs. This human-in-the-loop approach ensures precision, adaptability, and continuous improvement in addressing evolving litigation needs.

Ensuring Accuracy and Effectiveness

To validate our solutions, we use rigorous evaluation methods. Formative evaluations provide rapid feedback on system components, enabling continuous refinement. Summative evaluations assess the overall impact of our tools in real-world scenarios, such as claimant grid accuracy and time savings. By correlating these metrics, we ensure that improvements to specific components enhance overall system performance. This structured approach guarantees reliability and effectiveness in every review.

Conclusion: A Continuous Journey

Medical record review is an inherently complex and evolving task. By addressing its core challenges, adopting a holistic strategy, and rigorously validating our tools, we’re not just solving today’s problems—we’re building a foundation for future innovation.

“In the land of the blind, the one-eyed man is king.” 

Every improvement, no matter how small, is a step closer to making medical review faster, more accurate, and more effective.

For further reading, check out these related blogs about AI for Mass Tort Case Review:

• Using RAG to Avoid Hallucinations in AI for Mass Tort Case Review
• Introducing the Cyborg Paralegal
• Understanding Artificial Intelligence: A Primer for Mass Tort Attorneys