Tornevall Networks

← Back to docs

DNSBL V5 proxy checker

DNSBL V5 proxy checker

DNSBL V5 maintains an active registry of working proxies and current Tor exit nodes. Legacy DNSBL_V5.iplist is only used as a candidate source and is not the runtime model.

Scheduler

DnsblProxyServiceProvider registers:

php artisan dnsbl:proxy-sweep

every five minutes with overlap protection and background execution. Output is appended to:

storage/logs/dnsbl-proxy-sweep.log

The server must still run Laravel schedule:run from cron or another system scheduler.

A normal sweep:

  1. fetches and synchronizes the current Tor exit snapshot
  2. imports the next relevant legacy candidate batch
  3. checks the next ordinary proxy candidate batch

Use --no-tor-sync to skip the Tor snapshot and --no-import to skip the legacy import.

Proxy verification

An open TCP port is not enough. The checker actually connects through the candidate and fetches Tools' nonce-based endpoint:

/dnsbl/proxy-probe

Known endpoints are tried first, followed by the configured common ports. Results for each protocol and port are stored in dnsbl_proxy_endpoints, including status, latency, last check and last successful check.

Default ports:

80,443,1080,3128,8000,8080,8081,8118,8888,9050,9150

Override the list with DNSBL_PROXY_PORTS.

Failure threshold and cleanup

Ordinary proxies use:

DNSBL_PROXY_FAILURES_BEFORE_PURGE=1

This value is the number of consecutive failed verifications required before a candidate may be removed.

Behavior:

  • before the threshold, the row is marked failed and queued for retry
  • the existing DNSBL publication is left untouched while the failure count is below the threshold
  • when the threshold is reached, an explicit checker-owned delist is performed
  • the local row is deleted only after the DNSBL delist succeeds
  • a failed DNSBL mutation keeps the row as sync_error for retry

This prevents a temporary network failure from delisting a proxy before the configured threshold has actually been reached.

Proxy endpoint TXT metadata

When a proxy verifies successfully, the checker sends the actual working endpoint to the proxy-sync API. Each DNSBL owner receives a separate checker-managed TXT value containing IP, port, protocol and anonymity status.

Example:

Proxy verification: 203.0.113.10:8080; protocol=http; verified=yes; anonymous=yes; proxy_bitmask=130; proxy_flags=IP_CONFIRMED,IP_ANONYMOUS; schema=1; source=Tools proxy checker

IPv6 addresses are wrapped in brackets so the endpoint port remains unambiguous.

These records are separate from other source-registration TXT records. Proxy sync only replaces or removes TXT values beginning with Proxy verification: and leaves unrelated TXT records untouched.

TXT is supplemental metadata, not the DNSBL client contract. Clients that only support A records continue to use the bitmask exactly as before. When metadata is exposed through ToolsAPI, Tools should resolve and normalize the TXT information for the client; clients should not need to implement their own TXT parsing.

When a proxy reaches its purge threshold, its checker-managed proxy TXT is removed together with the checker-owned proxy flags.

Tor exit snapshot

Tor is handled as an authoritative feed rather than an ordinary proxy candidate. Every normal sweep refreshes the current export from Tor Project:

https://check.torproject.org/exit-addresses

A successful snapshot is used to:

  • add new exit IPs to dnsbl_proxy_hosts
  • mark them with tor_managed and refresh tor_last_seen_at
  • publish the Tor bitmask in dnsbl.tornevall.org and opm.tornevall.org
  • publish separate managed Tor exit: TXT metadata
  • preserve an ordinary working proxy on the same IP when it still has a verified endpoint
  • remove the Tor bit and Tor TXT when an IP disappears from a later successful snapshot
  • delete a Tor-only local row after successful DNSBL delisting when the exit is no longer present

Tor cleanup does not use the slower ordinary proxy failure threshold.

If Tor Project cannot be read, returns an invalid response, or the export contains no usable ExitAddress records, the snapshot is treated as unknown. No negative Tor cleanup is performed in that case, so a source outage cannot be interpreted as the entire Tor network disappearing.

Tor TXT example:

Tor exit: 203.0.113.20; current=yes; tor_bitmask=160; tor_flags=IP_SECOND_EXIT,IP_ANONYMOUS; schema=1; source=Tor Project

Bitmask

The checker uses current ToolsAPI semantics:

Result Mask
Working transparent proxy 2
Working anonymous proxy 130 (2 + 128)
Current Tor exit 160 (32 + 128)
Current Tor exit that also works as a proxy 162 (2 + 32 + 128)

Deprecated bit 1 is removed when the checker rewrites a record. Other active reputation bits are preserved.

DNSBL synchronization

The checker writes through:

POST /api/dnsbl/proxy-sync

The endpoint requires an active DnsblApiToken. Proxy synchronization is scoped to:

dnsbl.tornevall.org
opm.tornevall.org

It manages checker-owned proxy/Tor bits and checker-owned proxy/Tor TXT values without deleting unrelated reputation flags or unrelated TXT records.

Minimum production configuration:

DNSBL_PROXY_API_TOKEN=
DNSBL_PROXY_API_BASE_URL=https://tools.tornevall.net
DNSBL_PROXY_PUBLIC_BASE_URL=https://tools.tornevall.net

The token needs both add and delete permissions for automatic updates and cleanup.

Configuration

Common tuning values:

DNSBL_PROXY_STALE_AFTER_DAYS=7
DNSBL_PROXY_WORKING_RECHECK_HOURS=24
DNSBL_PROXY_FAILURE_RETRY_MINUTES=10
DNSBL_PROXY_FAILURES_BEFORE_PURGE=1
DNSBL_PROXY_IMPORT_BATCH_SIZE=2000
DNSBL_PROXY_CHECK_BATCH_SIZE=20
DNSBL_PROXY_CONNECT_TIMEOUT_MS=800
DNSBL_PROXY_REQUEST_TIMEOUT_MS=3500
DNSBL_PROXY_PORTS=80,443,1080,3128,8000,8080,8081,8118,8888,9050,9150
DNSBL_PROXY_TOR_SYNC_ENABLED=true
DNSBL_PROXY_TOR_TIMEOUT_SECONDS=10
DNSBL_PROXY_TOR_CACHE_MINUTES=30
DNSBL_PROXY_TOR_RECHECK_MINUTES=5

Choose DNSBL_PROXY_FAILURES_BEFORE_PURGE according to the desired tolerance. The threshold now controls both when a dead ordinary proxy may be delisted and when its local row may be deleted.

Public UI

The read-only registry is available at:

/dnsbl/proxies

It shows:

  • status badges for working proxy, failed check, Tor exit, anonymous, outdated and sync error states
  • real endpoints as protocol://ip:port
  • endpoint status and latency
  • failure count against the current purge threshold
  • current DNSBL bitmask
  • latest proxy check and latest Tor feed observation
  • location, ASN and network ownership from the shared MaxMind integration when available
  • source information, including authoritative Tor feed status

Tor-only rows do not need to expose a proxy port. They are present because the IP is a current exit node according to the authoritative feed.

Migrations

DNSBL V5 migrations live under:

database/migrations/dnsblv5

Tor synchronization additively introduces:

  • tor_managed
  • tor_last_seen_at

Migrations are forward-only and their down() methods are intentionally non-destructive.