Define
Agree the decision, population, economics, guardrails and counterfactual question.
How the analysis works
Replay the decision before you rewrite the rule.
Historical policy replay asks a controlled question: had a transparent alternative been applied to the same population, what would likely have changed in contribution profit—and what would have happened to the agreed guardrails?
The analytical spine
From decision logs to a governed challenger.
The method is deliberately legible. Each step makes assumptions, exclusions and evidence visible enough to challenge.
Reconstruct
Connect inputs, decisions, exceptions, offers, outcomes, costs and losses.
Replay
Measure the current policy and test bounded challenger rules on the same history.
Govern
Translate the result into implementable logic, monitoring and an approval path.
Illustrative comparison
One table leadership can interrogate.
This is an illustrative structure—not a client result. The final measures and thresholds are agreed with the institution.
| Measure | Current policy | Challenger | Interpretation |
|---|---|---|---|
| Approval rate | 62.4% | 63.1% | Within +1.0 pp guardrail |
| Expected loss | 4.8% | 4.8% | Stable |
| Contribution margin | 14.8% | 16.2% | +140 bps |
| Manual exception rate | 11.6% | 7.9% | Lower operational load |
| Policy form | Distributed rules | Six decision bands | Implementation-ready |
Profit means after consequence
Revenue alone is not the objective.
The metric design should prevent an apparent win from being created by moving risk, cost or operational burden elsewhere.
Economics
Start with a contribution-profit equation
Interest and fee income less funding cost, expected or realised loss, incentives and measurable operating costs.
Risk
Keep loss and policy constraints explicit
A challenger is not better if it creates value only by silently accepting a risk movement the institution would reject.
Operations
Account for capacity and treatment cost
Collections, manual reviews, exceptions and partner interventions have resource constraints that belong in the decision.
Governance
Document exclusions and stability
Time windows, missing data, policy changes and segment coverage are documented so the replay is not mistaken for certainty.
Typical data map
Enough history to connect choice and outcome.
The exact fields depend on the use case. A de-identified analytical extract is usually preferable to unnecessary personal data.
Decision
What was known and what was chosen?
Application or account attributes, scores, policy version, offer, price, limit, treatment, exception and timestamp.
Key requirement: reconstruct the actual decision state
Outcome
What happened after the choice?
Take-up, repayment, delinquency, loss, recovery, closure, repeat behaviour and sufficient outcome horizon.
Key requirement: consistent observation windows
Economics
Where did value accrue or disappear?
Income, funding cost, incentives, servicing cost, contact cost, agency cost, write-offs and recoveries where available.
Key requirement: an agreed profit definition
Governance
Which rules cannot simply be optimized away?
Eligibility, exposure, policy, fairness, risk-appetite, geographic and operational constraints.
Key requirement: named owners for guardrails
What this is—and is not
“A replay is evidence for a better decision conversation. It is not permission to skip validation, governance, controlled testing or human accountability.”
Method principle