DNSBL DMARC Intake and Review

This page describes the current DMARC flow in Tools for DNSBL-related review work.

It is not a normal end-user blacklist form. The intended callers are:

dnsbl-engine
internal helper jobs
site owners / admins reviewing DMARC data before deciding whether a reported source IP should be published to DNSBL

What the feature does

Tools can ingest DMARC aggregate reports and place them in an admin review queue instead of publishing directly from the incoming payload.

That queue lets an operator:

inspect the normalized report metadata
review one row per reported source IP
publish a selected row as spam (16)
publish a selected row as spam + fraud (84)
ignore a row without publishing it

This keeps the final blacklist decision manual.

API endpoint

`POST /api/dnsbl/dmarc/report`

The request uploads one DMARC payload for intake.

Supported payload forms include:

plain DMARC XML
gzip-compressed DMARC reports
ZIP-compressed DMARC reports
MIME-wrapped DMARC report payloads
forwarded or wrapped message/rfc822 mails that contain the DMARC XML/ZIP attachment inside nested MIME parts
notice-style / forensic DMARC mails that describe sender domain/IP/alignment/result in text and include copied headers

Request body

{
  "payload_base64": "H4sIAAAAA...",
  "source_type": "dnsbl_engine_dmarc",
  "source_name": "dnsbl-engine",
  "original_filename": "mailru-report.xml.gz",
  "content_type": "application/gzip",
  "content_encoding": "gzip"
}

Field notes

payload_base64 is required and must decode to a non-empty payload.
source_type is optional caller metadata.
source_name is optional human-readable caller metadata.
original_filename is optional and helps the reviewer identify the uploaded file.
content_type and content_encoding are optional hints about the original file format.

Authentication behaviour

The current DMARC intake path accepts these modes:

active DNSBL token through X-Dnsbl-Token or dnsbl_token
the token must be active and must have effective DNSBL add/write permission (can_add=true)
active DNSBL provider keys (provider=tornevall_dnsbl) are also accepted through the same token transport
active admin-owned Tools API keys are also accepted through the same token transport as full admin passthrough

Practical notes

Missing, invalid, or inactive DNSBL tokens now block DMARC ingestion with 401.
Active DNSBL tokens without add permission now fail with 403 reason="insufficient_dnsbl_scope".
Ordinary non-DNSBL Tools API keys, admin-session fallbacks, and anonymous internal DMARC uploads are no longer accepted for this endpoint.
Duplicate uploads are accepted as harmless no-ops and return duplicate: true with the existing report metadata.
That duplicate behavior is now also race-safe when two identical payloads reach the same unique payload_hash at nearly the same time; callers now still receive the existing report instead of a raw SQL duplicate-key failure.
Invalid or unparsable payloads return 422.
When dnsbl-engine upload is denied, the local DMARC mail/file is kept in the spool for retry instead of being deleted.

Typical success response

{
  "ok": true,
  "duplicate": false,
  "message": "DMARC payload ingested for admin review.",
  "report": {
    "id": 12,
    "report_id": "33491921110551199191685577600",
    "org_name": "Mail.Ru",
    "policy_domain": "tornevall.net",
    "status": "pending",
    "record_count": 1,
    "source_ip_count": 1,
    "date_begin": "2023-06-01T00:00:00+00:00",
    "date_end": "2023-06-02T00:00:00+00:00",
    "created_at": "2026-04-19T21:03:00+00:00"
  },
  "records": [
    {
      "id": 99,
      "source_ip": "60.13.8.218",
      "message_count": 1,
      "disposition": "reject",
      "dkim_result": "fail",
      "spf_result": "fail",
      "recommended_action": "spam_fraud",
      "status": "pending"
    }
  ]
}

Review queue in the web UI

After intake, the report is available in:

/admin/dnsbl/dmarc

From there an admin can open the report details page and inspect:

report ID
sending organization
policy domain
DMARC policy fields (p, sp, adkim, aspf, pct)
report time window
report kind (Aggregate XML or Forensic notice)
normalized report content (XML for aggregate reports, normalized notice text + copied headers for forensic-style reports)
one row per reported source IP
reverse-DNS hostnames resolved from those source IPs when PTR data exists

Each source-IP row exposes these actions:

Publish spam → publishes DNSBL bitmask 16
Publish spam+fraud → publishes DNSBL bitmask 84
Ignore → leaves the IP unpublished and marks the row as handled in the queue

Status values you may see

Report status

A DMARC report can appear with statuses such as:

pending - no rows reviewed yet
partial - some rows handled, some still pending or mixed
reviewed - rows were handled with a mixed published/ignored outcome
published - every row ended up published
ignored - every row ended up ignored

Record status

A single source-IP row can appear as:

pending
published
ignored
failed

When a publish attempt has been made, the UI also shows the last stored publish result for that row.

Error behaviour

Typical failure cases:

401 missing_dnsbl_token - no DNSBL token was supplied
401 invalid_dnsbl_token - the supplied value did not resolve to an active DNSBL token
401 inactive_dnsbl_token - the DNSBL token exists but is not active
403 insufficient_dnsbl_scope - the DNSBL token is active but does not have add/write permission
403 wrong_token_type - the supplied value matched another Tools token type instead of a DNSBL token
422 invalid_payload - payload_base64 did not decode into a usable payload
422 invalid_dmarc_report - the uploaded payload could not be parsed as a DMARC report

Duplicate uploads do not fail. They return a normal success response with duplicate: true.

Recommended usage pattern

Collect or receive a DMARC aggregate report.
Base64-encode the raw payload.
Upload it to POST /api/dnsbl/dmarc/report with an active DNSBL token that has add permission.
Open /admin/dnsbl/dmarc.
Review the reported source IP rows.
Publish only the rows that should really become DNSBL entries.

Tornevall Networks