How to diff a sanctions list

Every sanctions-screening program eventually asks the same question: "did anything change since the last time we pulled this list?" The naive answer — compare today's file to yesterday's file, line by line — breaks in practice, for reasons that have nothing to do with effort and everything to do with how issuing authorities actually publish and revise their lists. Diffing a sanctions list correctly is a specific, learnable discipline, and getting it wrong produces two opposite failure modes: missed new designations, and stale true-positive flags on people or entities who were removed months ago.

Why a naive text diff fails

Sanctions lists are not append-only logs. Between any two snapshots, records can:

• Be added — a new designation.
• Be removed — a delisting, license, or judicial reversal.
• Be amended in place — a corrected date of birth, a newly disclosed alias, a program reclassification — without the underlying person or entity changing at all.
• Be renumbered or re-identified — the same real-world entity gets a new record ID because the issuing authority migrated systems, restructured its list, or merged two legacy lists into one.

A line-by-line text diff conflates all four. It reports an amendment as a delete-plus-add, misses a renumbering entirely (or worse, reports it as a removal and a fresh addition for an entity that was never actually delisted), and gives no signal about which changes are urgent.

The identifier problem is the real problem

The hardest part of diffing isn't detecting that something changed — it's proving that record A in yesterday's snapshot and record B in today's snapshot are (or aren't) the same real-world entity. Most issuing authorities assign a persistent identifier to each record specifically so downstream users can track it across revisions, but that identifier is not guaranteed to survive a platform migration.

The UK is a live example. Financial-sanctions designations moved from OFSI's legacy Consolidated List to the FCDO-published UK Sanctions List on 28 January 2026, and newly designated persons are now assigned a "Unique ID" rather than the old "OFSI Group ID" (GOV.UK guidance). A screening pipeline that keyed its match history to the OFSI Group ID and didn't account for the identifier-scheme change risks treating every pre-existing UK designation as a new entity the day it re-ingests the successor list — a false signal at exactly the moment the underlying regime didn't actually change. The reverse mistake is just as real: silently mapping old IDs to new ones without verifying referent continuity can just as easily merge two genuinely distinct entities.

What to diff — four categories, not one

Treat every list refresh as producing four distinct outputs, not a single "changed" bucket:

1. True additions. A record with no plausible match in the prior snapshot. This is the case everyone builds for — it's also the easiest to get right, because there's nothing to reconcile against.
2. Removals (delistings). A record present in the prior snapshot, absent from the current one, with no successor record. This is the category screening programs most often under-build. Our own June enforcement digest covered a concrete example: OFAC removed a set of Russia-related SDN designations on 24 June 2026 in the same month it added new ones. A pipeline that only ingests additions and never processes delistings accumulates stale true-positive flags — customers who cleared their sanctions exposure months ago keep tripping alerts, which erodes trust in the whole screening program and buries the alerts that are still live.
3. In-place amendments. Same identifier, same entity, changed fields — a corrected alias, an added identifier, a program tag change. These need a field-level diff, not a record-level one, because the significance varies: an alias correction is informational, but a program reclassification (say, from a sectoral list to a full blocking designation) can change what's legally required of a counterparty holding that name in their book.
4. Identifier remapping. The UK case above — the entity is continuous, but its persistent ID changed because of an upstream platform or list-structure change. This category is invisible to both a naive diff and a pure identifier-match diff; it requires verifying continuity through secondary signals (name, other identifiers, designation date) when the primary ID changes.

A workable pattern

The reliable approach treats every list as a versioned corpus, not a file to overwrite:

• Keep every historical snapshot, each stamped with its as-of date and (where the authority publishes one) its own version identifier.
• Diff at the field level within a matched identifier, not just at the record level, so an amendment is legible as "what changed" rather than a delete-and-recreate.
• When an identifier disappears, actively check for a successor before recording it as a removal — a name-and-attribute match against the new snapshot, not just an absence check.
• Log removals as first-class delist events with a timestamp, not silent deletions — the removal itself is part of the audit trail a compliance team needs to explain why a name that used to be a hit no longer is.

That last point is the connective thread back to why this matters at all: a screening hit is only as defensible as its lineage — which authority, which list, which version, as of when. The same discipline applies in reverse to a non-hit that used to be a hit. If you can't show when and why a name stopped matching, "we don't screen against that anymore" is not an answer a regulator will accept on faith.

See it in practice

Our recent-designations timeline shows additions as they land, per source, with dates. Several sources also have their own per-source timeline view for isolating one list's activity. The source catalog documents each list's issuing authority, format, and update cadence — the starting facts you need before building a diff pipeline against it. And browsing a source directly shows the current state you're diffing against, one record at a time.