Document review is one of the tasks where plausible smoothing is more dangerous than outright refusal. When sources disagree, the useful response is to surface the disagreement clearly so a human can decide what to do next. The tempting response is to produce a coherent narrative that quietly resolves the tension. I constructed a controlled review packet that contained several deliberate conflicts and ran it repeatedly through Opus 5.2 under consistent instructions. The model produced fluent, professional-looking reviews. It also systematically missed or softened certain classes of conflict. The misses were not random. They followed patterns that now shape how I design review workflows and how I brief human reviewers.
Why Conflicting Evidence Is a High-Value Stress Test
Real Professional Packets Rarely Arrive Clean
Client documents, internal strategy packs, and due-diligence materials frequently contain chronological inconsistencies, overlapping but non-identical claims, and recommendations that sit at an angle to earlier constraints. A review process that only works on clean material is of limited practical use.
Smoothing Errors Travel Farther Than Obvious Failures
An output that invents a fact is often caught by a moderately careful reader. An output that erases a conflict by choosing one side and omitting the other can pass a quick scan and then shape downstream decisions. The second error is harder to detect and therefore more costly.
The Test Isolates a Specific Failure Mode
By holding the instructions constant and embedding known conflicts, the exercise reveals whether the model tends to preserve tension or to resolve it. That behavioral tendency is more actionable than a generic accuracy score.
Construction of the Conflicting Review Packet
Types of Conflict Deliberately Embedded
The packet contained four categories of tension: chronological inconsistency between two dated statements, quantitative disagreement on a key performance figure, overlapping but non-identical descriptions of a customer segment, and a final recommendation that partially contradicted an earlier hard constraint. Each conflict was unambiguous on close reading yet easy to miss under time pressure.
Instructions Given to the Model
The model was asked to produce a structured review that included a summary of key claims, an explicit list of internal inconsistencies or unresolved tensions, and a cautious assessment of the strength of the overall evidence. The instructions emphasized that unresolved conflicts should be listed rather than resolved.
Controlled Repetition
The same packet and instructions were used across multiple independent runs. Outputs were compared for whether each embedded conflict appeared in the inconsistency list, whether it was softened in the summary, and whether the final assessment acknowledged the resulting uncertainty.
What the Model Consistently Surfaced
High-Level Structure and Explicit Claims
Across runs the model reliably captured the main sections and the most prominent quantitative and qualitative claims when those claims did not conflict with anything else. Fluency and organization remained high.
Some Surface-Level Chronological Notes
Mild date discrepancies that appeared in close proximity were occasionally noted. The detection rate dropped when the conflicting statements were separated by several pages or appeared in different document sections.
Open Questions When Explicitly Forced
When the instructions strongly required a dedicated inconsistencies section, the model produced more complete lists than when the requirement was softer. Instruction strength measurably affected detection completeness.
What the Model Consistently Missed or Softened
Quantitative Conflicts Presented as Ranges or Averages
When two sources offered different figures for the same metric, the model sometimes produced a single smoothed value or a vague range that erased the disagreement. The original conflict disappeared from both the summary and the inconsistency list in a subset of runs.
Segment Descriptions That Overlapped but Did Not Match
Customer or market descriptions that shared some attributes while differing on others were frequently collapsed into a single coherent portrait. The non-matching attributes were dropped rather than flagged.
Recommendation-to-Constraint Tension
The mild contradiction between an earlier hard constraint and a later recommendation was the most frequently softened item. The model often restated the recommendation with reduced force or omitted the constraint from the final assessment, producing an artificially consistent narrative.
Conflicts Separated by Distance in the Source
Tensions whose elements appeared far apart in the packet were missed more often than those whose elements appeared close together. Proximity in the source material influenced detection probability.
The following table records the conflict categories and the observed detection behavior under the tested conditions:
Conflict Type | Typical Model Behavior | Professional Risk if Missed |
|---|---|---|
Chronological inconsistency | Detected when close; missed when distant | Incorrect timeline assumptions |
Quantitative disagreement | Often smoothed into range or single value | Flawed calculations or forecasts |
Overlapping but non-identical segments | Collapsed into coherent portrait | Misleading market or customer understanding |
Recommendation vs earlier constraint | Softened or constraint dropped | Advice that violates stated limits |
Practical Controls That Reduce the Damage
Force an Explicit Inconsistencies Section With Examples
Providing one or two short examples of the kind of tension that must be listed improves detection rates. Vague instructions to “note any problems” are less effective than concrete illustrations of chronological, quantitative, and logical conflicts.
Require Source Location Pointers on Every Claimed Inconsistency
When the model must point to the locations of both sides of a conflict, invention and smoothing become harder. Location requirements also accelerate human verification.
Treat the Final Assessment as Draft Language Only
Any concluding judgment about overall evidence strength is reviewed with the explicit question: “Does this assessment preserve the tensions that the inconsistency list identified?” If the answer is no, the assessment is rewritten.
Sample Distant Conflicts During Human Review
Because distance in the source reduces detection probability, reviewers should deliberately check for tensions whose elements are widely separated. This targeted sampling is more efficient than exhaustive re-reading.
Implications for Document-Review Workflow Design
Detection Completeness Is Instruction-Sensitive
The same model under the same source material produced noticeably different inconsistency lists when the instructions varied in strength and concreteness. Instruction design is part of reliability, not an afterthought.
Smoothing Is a Default Tendency Under Narrative Pressure
When the output format encourages a coherent summary, the model resolves tension more readily. Formats that isolate the inconsistency list before any narrative summary reduce the pressure to smooth.
Human Review Must Be Calibrated to the Known Miss Classes
Reviewers who know that quantitative conflicts and distant tensions are frequently missed can allocate their attention accordingly. Generic “check for accuracy” guidance is less effective than guidance that names the high-risk miss categories.
Closing the Conflicting-Evidence Failure Report
In a controlled document review containing deliberate conflicts, Opus 5.2 reliably captured non-conflicting claims and structure while systematically missing or softening certain classes of tension—especially quantitative disagreements, overlapping segment descriptions, recommendation-constraint mismatches, and conflicts whose elements were widely separated in the source. Stronger, more concrete instructions improved detection but did not eliminate the patterns.
Useful remains a higher bar than impressive. A document-review workflow earns its place when unresolved tensions remain visible rather than being quietly resolved into fluent prose. Designing instructions, output structure, and human review around the known miss classes is how that bar becomes reachable in practice.