In late 2025, I started building a working product demonstration showcasing a common healthcare use case: reconciling prior authorizations with financial claims. This project was based in part from my work on a medical pharmacy modernization initiative.
In projects like this, I find that expressing my thought process and related exploration in words is a useful way to help me learn and stay focused. Given that some of this exploration required the use of AI, I decided to keep these build journals offline to avoid introducing unnecessary “noise” on this blog.
However, I still think there is value in sharing a few of the ChatGPT summaries from these engineering journal entries for relevant context and future reference. You can access the project portfolio page here.
November 18, 2025 - Project Foundation & First Agent
The project officially began with a deliberate decision to consolidate effort into a single integrated medical pharmacy initiative combining architecture, code, presentation material, and operational narrative. The user organized the effort into six chapters covering business capabilities, business assessment findings, value creation levers, agentic architecture, a deep dive into Auth-to-Claim Match (ATCM), and future considerations. Drawing on prior healthcare consulting experience, the user narrowed focus to the ATCM process because it represented a high-friction operational reconciliation layer involving prior authorizations, claims adjudication, manual review, and complex client-specific rules. The day was heavily grounded in business decomposition, operational pain-point analysis, and domain mapping across utilization management, claims integrity, and clinical governance workflows.
From a technical perspective, the user generated synthetic healthcare datasets and implemented a deterministic Python matching engine capable of identifying clean matches, quantity mismatches, date mismatches, provider mismatches, and missing authorizations. The user then designed a layered agentic architecture separating deterministic systems from reasoning systems, concluding that LLMs should not perform authoritative matching logic but instead provide interpretation and operational guidance. This led to the successful implementation of an initial CrewAI-based “Exception Explanation Agent” that translated deterministic outputs into reviewer-friendly narratives and routed claims to specialist investigative paths such as authorization discovery and date/window validation.
The focus shifted toward orchestration design, deterministic post-processing, and tool integration. While experimenting with CrewAI tools and push notifications, the user discovered an important operational reality: LLMs cannot be relied upon to invoke tools consistently. In response, the architecture was redesigned so that agents generated structured JSON outputs while deterministic Python logic controlled all operational side effects, including push notifications. This represented a key architectural refinement separating semantic interpretation from transactional execution and reinforced the broader principle that deterministic systems should retain operational authority wherever possible.
November 19, 2025 - Deterministic Validation Design
The remainder of the day focused on expanding the deterministic Date/Window Validation capability. The user identified several real-world healthcare timing nuances missing from the original model, including grace periods, retroactive authorization eligibility, inactive authorization statuses, and operational aging logic. Synthetic datasets were expanded with new fields supporting retro-auth and timing evaluation, and the user designed a deterministic validation flow capable of classifying claims as valid, retro-eligible, expired, or invalid. After evaluating whether this logic belonged inside CrewAI, the user concluded that date validation should remain outside the LLM framework entirely because the rules were explicit, auditable, and computationally deterministic.
November 20, 2025 - Authorization Discovery Exploration
The user validated the retro-authorization and grace-period calculations after generating expanded datasets containing retro-auth and timing fields. Through testing several realistic scenarios, the user confirmed that the deterministic date-validation logic could correctly identify valid grace-period claims and retro-eligible authorizations. Rather than immediately reintegrating the logic into the main application, the user intentionally paused implementation to focus attention on a more advanced third agent: the Authorization Discovery Agent, which would require more sophisticated reasoning and investigative capabilities.
The remainder of the day was devoted to understanding why missing authorizations occur and how investigative search logic should work. The user identified identity normalization as a foundational challenge and introduced the concept of a “member ID crosswalk” to reconcile different member identifiers across claims, authorization, and eligibility systems. Using concrete claim examples, the user manually explored how timing windows, retro-auth rules, drug mappings, and cross-system identities could explain failed matches. This led to an important architectural realization: deterministic systems should generate candidate authorizations and operational clues, while LLMs should reason over those findings to explain why reconciliation failed and recommend next investigative actions.
November 21, 2025 - Tiered Search & Reasoning
The Authorization Discovery Agent evolved from a conceptual investigative idea into a structured search and ranking framework. The user reframed the problem away from exact matching and toward “diagnostic proximity,” recognizing that the deterministic engine had already proven no direct authorization existed. To investigate plausible operational explanations, the user designed a tiered search methodology using progressively relaxed constraints — such as widening date windows, relaxing drug matching, introducing alternate member IDs, and examining provider-based clues. Each tier was assigned a confidence score, effectively encoding healthcare domain expertise into a deterministic candidate-ranking system inspired by Progressive Constraint Relaxation (PCR).
The user then operationalized this framework using expanded synthetic datasets and exploratory Colab workflows. Several claims lacking authorizations were manually evaluated across multiple search tiers, producing investigative summaries that surfaced alternate member identities, plausible candidate authorizations, and operational gaps. By the end of the day, the project had evolved into a layered hybrid architecture combining deterministic reconciliation, heuristic candidate discovery, structured scoring, and LLM-assisted reasoning. The user concluded that the next step would be modularizing each search tier into reusable functions and integrating OpenAI-generated investigative guidance programmatically.