Automate the Noticing, Not the Finding

Most articles about automating security audits hand you a script that shells out to nmap and prints the result. That is not automation, it is a slower way to run nmap.

The thing worth automating is not finding problems — dedicated scanners do that better than anything you will write. It is noticing change. A port that was not open last week. A certificate that will expire during your holiday. A DNS record pointing somewhere nobody remembers creating. Those are cheap to detect, expensive to miss, and no commercial tool knows what your estate is supposed to look like.

The architecture, which is the whole idea

Every check in this article is the same four steps.

Collect the current state. Normalise it into sorted, stable JSON. Commit it to a git repository. Let git diff tell you what moved.

That last step is the trick. You do not need a database, a dashboard or a diffing engine — you need a file that changes only when reality changes, and a tool that is extremely good at showing you how files changed. The commit history becomes an audit trail you did not have to build, with timestamps and authorship, and git log -p answers "when did this appear" in one command.

The only discipline it requires is that your output is deterministic. Sort every list, drop timestamps from the payload, and round anything that jitters — otherwise every run produces a diff and you stop reading them.

import json, pathlib, subprocess

def snapshot(name: str, data) -> None:
    """Write a stable, sorted JSON snapshot and commit only if it changed."""
    path = pathlib.Path("snapshots") / f"{name}.json"
    path.parent.mkdir(exist_ok=True)
    path.write_text(json.dumps(data, indent=2, sort_keys=True) + "\n")

    subprocess.run(["git", "add", str(path)], check=True)
    changed = subprocess.run(["git", "diff", "--cached", "--quiet"]).returncode == 1
    if changed:
        subprocess.run(["git", "commit", "-m", f"{name}: state changed"], check=True)
        print(f"CHANGED: {name}")

Certificates, before they expire on a Saturday

Do not shell out to openssl and parse the text. The cryptography library reads the certificate properly, and you get real objects instead of a regex.

import socket, ssl
from datetime import datetime, timezone
from cryptography import x509
from cryptography.hazmat.primitives import hashes

def inspect_cert(host: str, port: int = 443) -> dict:
    ctx = ssl.create_default_context()
    with socket.create_connection((host, port), timeout=10) as sock:
        with ctx.wrap_socket(sock, server_hostname=host) as tls:
            der = tls.getpeercert(binary_form=True)
            protocol = tls.version()

    cert = x509.load_der_x509_certificate(der)
    expires = cert.not_valid_after_utc
    sans = cert.extensions.get_extension_for_class(
        x509.SubjectAlternativeName).value.get_values_for_type(x509.DNSName)

    return {
        "issuer": cert.issuer.rfc4514_string(),
        "expires": expires.isoformat(),
        "days_left": (expires - datetime.now(timezone.utc)).days,
        "sans": sorted(sans),
        "sig_algorithm": cert.signature_hash_algorithm.name,
        "fingerprint": cert.fingerprint(hashes.SHA256()).hex(),
        "tls_version": protocol,
    }

The field people forget to record is the fingerprint. Expiry warnings are the obvious value, but a fingerprint that changes when you did not reissue anything is a much more interesting event — it means someone else renewed your certificate, or you are not talking to who you think you are.

The DNS check that earns its keep

Dangling CNAMEs are the best return on effort in this entire article. Someone creates promo.example.com pointing at a cloud bucket or a SaaS product, the campaign ends, the resource is deleted, the DNS record is not. Anyone who claims that resource name now serves content on your domain, with your cookies in scope.

import dns.resolver

TAKEOVER_SUFFIXES = (
    ".s3.amazonaws.com", ".cloudfront.net", ".azurewebsites.net",
    ".github.io", ".herokuapp.com", ".pages.dev",
)

def check_cname(name: str) -> dict | None:
    try:
        target = str(dns.resolver.resolve(name, "CNAME")[0].target).rstrip(".")
    except dns.resolver.NoAnswer:
        return None                       # no CNAME, nothing to dangle
    except dns.resolver.NXDOMAIN:
        return None

    risky = target.endswith(TAKEOVER_SUFFIXES)
    try:
        dns.resolver.resolve(target, "A")
        resolves = True
    except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer):
        resolves = False

    # A CNAME at a third-party host that no longer resolves is the danger case
    return {"name": name, "target": target,
            "dangling": risky and not resolves}

Run it over every name in your zone. A dangling result is not automatically a takeover — the provider may reserve names, and behaviour differs by service — but it is always worth a human looking, and it is exactly the thing that never gets noticed otherwise.

Headers, and why you check them from outside

A security-header check is five lines, and the reason to automate it is that headers get removed by accident. Someone adds a proxy, someone changes a config, and Strict-Transport-Security quietly stops being sent on one route.

import httpx

EXPECTED = {
    "strict-transport-security", "x-content-type-options",
    "referrer-policy", "content-security-policy",
}

def check_headers(url: str) -> dict:
    r = httpx.get(url, follow_redirects=True, timeout=15)
    present = {k.lower() for k in r.headers}
    return {
        "url": str(r.url),
        "status": r.status_code,
        "missing": sorted(EXPECTED - present),
        "server": r.headers.get("server", ""),
    }

Check the error pages too, not only the home page. Headers set with nginx's add_header without the always flag disappear on 404s and 500s — which is precisely where a reflected error message might need a Content Security Policy.

Ports, against a baseline you wrote down

Do not reimplement a port scanner. Run the real one and normalise its output — nmap -oX gives you XML that parses cleanly, and you get the accuracy of a tool that has had twenty years of edge cases beaten out of it.

The part that matters is the comparison. An open port is not a finding; an open port that is not on your approved list is a finding. Keep the baseline in the same repository as the snapshots, as a plain file someone can edit with a comment explaining why each entry is there.

approved = {"tcp/22", "tcp/80", "tcp/443"}
observed = parse_nmap_xml("scan.xml")          # -> {"tcp/22", "tcp/443", "tcp/6379"}

unexpected = sorted(observed - approved)        # ['tcp/6379']  <- a Redis on the internet
gone       = sorted(approved - observed)        # things that stopped answering

The gone set is worth alerting on as well. A service that stopped listening is either a change you meant to make, or an outage nobody has noticed yet.

What not to automate

Three things, because the failure mode of enthusiasm here is an alerting channel everyone mutes.

Judgement. A script can tell you a header is missing. It cannot tell you whether that matters on that route. Automate the collection, keep the decision with a person.

Anything noisy. If a check fires on normal variation, it will be ignored within a fortnight and then it is worse than not having it, because its silence now means nothing. Tune it until a notification is genuinely unusual.

Vulnerability discovery. Your script will not out-perform a maintained scanner. Use one, and spend your effort on the things that are specific to your estate — the approved-port list, the zone file, the certificate inventory. That knowledge is what no vendor has.

Running it

A scheduled CI job is usually the right home: it has credentials management, a log, and somewhere to send a notification. Run the read-only checks — certificates, DNS, headers — as often as you like. Run anything that touches a network with packets on a schedule your team knows about.

And scope it explicitly to assets you own, in a list that lives in the repository rather than in an argument. Scanning something by accident because a wildcard resolved further than you expected is a bad afternoon, and the defence is a written scope your script reads rather than a CIDR range you typed.

Scope and permission

Everything above is intended for infrastructure you are responsible for. Port scanning, TLS probing and DNS enumeration against systems you do not own may be unlawful depending on jurisdiction, and is against the terms of most hosting providers regardless.

Library APIs change: cryptography moved from not_valid_after to not_valid_after_utc and the older spelling is deprecated. Check the versions you install rather than trusting these snippets. Subdomain takeover behaviour varies by provider and changes as providers add reservation mechanisms, so treat a dangling CNAME as a prompt to investigate rather than a confirmed finding.