Dataset Selection & Reference

The appliance's data snapshot bundles every dataset ProofAML publishes. The DATASETS environment variable controls which of them are actually loaded into memory and screened against:

# Load everything in the snapshot
-e DATASETS=all

# Load only specific datasets (comma-separated ids)
-e DATASETS=us_ofac_sdn,us_ofac_cons,un_sc_sanctions,eu_fsf,gb_hmt_ofsi,gb_fcdo_uksl,ca_dfatd_sema_sanctions

• all (the default) loads every dataset in the snapshot.
• A comma-separated list of dataset ids restricts loading to exactly those datasets — useful to keep memory footprint down, or to scope screening to a jurisdiction's regulatory requirement (e.g. sanctions-only).
• The ids you can use are exactly the slug values in the reference table below — the same ids returned in /v1/status's datasets[].id, and the same values accepted in a /v1/search or /v1/batch request's datasets field to scope an individual query further, within whatever DATASETS already loaded.
• Restricting DATASETS is a load-time decision (it requires a container restart to change); the per-request datasets field in /v1/search is a query-time filter within whatever is loaded — you can't query a dataset at request time that DATASETS excluded at load time.
• Selection is entity-level, not dataset-level. An entity loads if it belongs to any requested dataset, and it keeps all of its co-listings — so /v1/status's datasets list can include ids you didn't request (e.g. DATASETS=us_ofac_sdn,eu_fsf can still surface un_sc_sanctions or icij_offshoreleaks in /v1/status, because a loaded entity is also designated there). Entities that belong only to datasets you didn't request are fully absent — never partially loaded.

Dataset families

Every dataset belongs to one of five families, matching the families published at /sources/:

• Sanctions & watchlists — binding legal designations from a governmental or intergovernmental authority (OFAC, UN, EU, UK, Canada).
• Politically exposed persons (PEP) — office-holder rosters used for enhanced due diligence, not a prohibition list.
• Law enforcement & wanted — fugitive/most-wanted lists and enforcement press releases; an unproven allegation, not a conviction.
• Investigative journalism — leak-derived corroboration data (e.g. ICIJ Offshore Leaks); backs cross-references, not sanction/PEP/wanted matches.
• Business registries & ownership (KYB) — company and beneficial-owner registries, for screening a business and its ultimate beneficial owners alongside named individuals.

Reference table

Dataset id, title, issuing authority, and a link to the corresponding public source page (license, current list version, and entity count) for every dataset the appliance can load. This table is generated from the same source catalog that backs /sources/, so it always matches what the appliance actually ships.

Sanctions & watchlists

Politically exposed persons (PEP)

Law enforcement & wanted

Investigative journalism

Business registries & ownership (KYB)

See also

• Configuration reference — the full DATASETS entry alongside every other environment variable.
• Search API reference — the per-request datasets field.
• Updates & synchronization — datasets are updated independently, one delta file per dataset.