I ran an AI pentest on my homelab — and misread my own results

Update — 2026-09-20: the findings from this sweep are fixed, deployed, and live-verified.

The HIGH I opened this post with — the unauthenticated SSRF in my LLM gateway — is closed. The fix landed in two commits. 3034f3e puts a destination guard on the provider test/discover routes (_validate_api_base_destination(): scheme http/https only, host has to resolve, every resolved address has to be public unless it is in an explicit CIDR allowlist). 196a3e8 closes the other three findings from the same scan: a crypto key loader that returned None on an I/O error, an auth path that let requests through when the key-manager lookup failed, and /health echoing the internal provider map. 31 new tests, full suite 2312 passing.

I re-ran the SSRF probes before writing this. 127.0.0.1, 169.254.169.254, the LAN addresses and localhost now all return HTTP 400 with a reason, and a legitimate public provider base still passes the guard through to upstream auth.

Two caveats I am not going to smooth over. The fourth finding is fixed scoped, not fully: /health no longer leaks the private endpoint map — that was the highest-value disclosure — but the cost/plugin/export JSON endpoints are still unauthenticated, deliberately, because the admin UI fetches them without auth and gating the API alone would break the dashboard without protecting anything. The real fix there is a login for the console, and that is a decision, not a patch.

And a correction to my own correction. The System block below called the scan brain a 38B MoE in NVFP4. It is not 38B. Qwen3.8-Flash-Next is a 180 B-parameter MoE — 512 experts per layer, 10 active per token, 48 layers — and what I serve is Mia's 34-shard NVFP4 re-shard (attention o_proj in MXFP8), 105.8 GB on disk. I am quoting 180 B because two independent counts agree: Hugging Face reports 179,999,981,459 parameters for the base model, and unpacking the 4-bit tensors in the quantized shards and adding them up lands on 180.00 B. "38B" was me reading the "3.8" in the model name as a size. (My first pass at this correction said 103.7 B — that counted stored bytes, quantization scales included, as parameters. Same class of mistake, smaller number. Better to write that down than to quietly publish the second guess.) The scan brain is the same box as the Qwen3.8-Flash-Next performance saga.

Update — 2026-09-14: the summary I published was wrong, and my own tooling is what fooled me.

I've retitled this post. It went out on September 9 as "It found almost nothing", because that is what my sweep summary said. The summary was wrong.

On September 10 I re-ran the whole 38-target sweep after fixing the token handling. It completed cleanly this time — driver exit 0, 00:16 to 05:05 UTC, all 38 targets — and FINDINGS.md again reported zero findings for every target. That second uniform zero is what finally made me look. If every target burned 1.5–7M tokens of real work, "nothing found, everywhere" was not a result. It was a bug.

It was a bug. parallel_driver.py matched the run directory as strix_runs/<slug>/ — with a trailing slash — while the Strix banner prints that path without one. run_dir never matched, the per-target parse silently no-op'd, and every target got summarized as clean. One character.

Recounted line by line from the run logs and strix_runs/:

The two HIGHs:

• An unauthenticated SSRF in my LLM gateway. The two provider test/discover endpoints take a caller-supplied api_base and fetch it with no scheme or host restriction, and both routes dispatch before the auth check — so no API key is needed, and the internal response is parsed and echoed back to the caller. CWE-918, CVSS 7.5. That is the service that fronts every other model call in the house. The fix task was written that same day; "it's LAN-only" is an explanation, not a mitigation. (It shipped on 09-20 — see the update at the top of this post.)
• A container running as UID 0 in the stock-analysis app.

The rest is thirty-one MEDIUM findings, plus two LOW and one informational — Homebox, SearXNG, IT-Tools, Trilium, Uptime Kuma, UrBackup and others. Four remediation tasks came out of it.

So what actually went wrong? Two separate things, and I only published one of them:

1. Round one failed for a real reason. Unbounded subagent fan-out: a single target burned 21.78M tokens and runs hit the 2–3h wall. Nothing about that part of the story is wrong.
2. The conclusion I drew from it was an instrument artifact. "The local model can't find anything" was not a measurement of the model. It was a measurement of my parser.

The honest revised bottom line: on this local model, Strix does find real vulnerabilities — including a HIGH in my own gateway — at 2–7M tokens per target. What it took was a harness I had actually tested. I trusted the summary instead of the raw logs, and that is the mistake worth writing down.

Everything below this line is the post as it was published on 2026-09-09, kept as written — including the conclusion the 09-10 re-run disproved. The corrected numbers are in the update at the top. The two things I fixed in place are the model spec and the target counts in the System block, because those were wrong facts rather than wrong conclusions.

I spent three days this week running an AI pentesting agent against my own homelab, "everything reachable" scope, which turned out to be 38 targets. Strix is the tool, open source, agentic exploit + PoC validation, and I ran it with a local model instead of the cloud frontier models the docs push. Figured I'd share the numbers, including the one I did not expect: 35 of 38 targets came back with zero findings, and the findings that did land came exclusively from runs I killed, never from runs that finished.

System

• Qwen3.8-Flash-Next on a DGX Spark — a 180 B-parameter MoE (512 experts, 10 active per token) — served through my LLM gateway as the 34-shard NVFP4 re-shard in prod
• Strix v1.6.2, quick mode, READ-ONLY instruction ("do not modify, delete, damage or DoS anything; validate with minimal reversible requests")
• 38 targets: 5 app source repos, 4 public endpoints, 15 LAN app hosts behind the internal reverse proxy, 13 direct-IP targets (docker app ports, the gateway, the inference box, Proxmox guests), 1 LAN monitoring host
• No turn cap in rounds one and two; the re-run that produced the corrected table above used --max-turns 30 per target, with a 2-3h wall budget and 4 scans in parallel

Round one: scope drama First dispatch was 14 hand-picked targets. That got rejected within a few hours — the expectation was the full reachable surface, not a curated subset. Killed round one, rebuilt the list from the wiki's inventory snapshot, and live-probed every candidate with /dev/tcp before writing the targets file. That step killed 5 dead entries before they polluted the run and caught a split-horizon DNS leak where 3 LAN hostnames resolved to my public IP instead of the internal box. Cheap step, worth doing before every sweep.

Round two: serial was unusable I had the L1 agent act as scheduler, spawning one scan at a time. One target per hour. At that rate the remaining 31 targets would take roughly 28 hours. Killed it the next morning: 7 targets done overnight.

Round three: a parallel driver Wrote a small parallel driver instead. Four workers, each atomically claims the next undone target via lock files, runs Strix with the READ-ONLY instruction, appends a findings block, marks the target done. Crash-resumable, so a dead worker never re-scans finished work. It chewed through the remaining targets in about a day and a half.

The numbers

The 19 findings on the LAN/public side break down as mostly "no issue identified" coverage items with a few real leads: unauthenticated GET /api/revalidate on my homepage (cache manipulation), open self-registration plus exposed swagger on homebox, and on the attachments tool a reachable PocketBase admin UI plus an OAuth credential-disclosure endpoint that returns 200. All worth fixing regardless of what the tool says.

The part that annoyed me: every one of those 19 findings came from an interrupted run. The scans that completed cleanly found nothing, every time. Deep scans burned 5M+ tokens on recon spirals and hit the 2-3h wall with nothing to show. The runs that produced anything were the ones my own tooling killed mid-flight. The forum wisdom of "just give it more turns, it'll converge" is wrong on this model. I gave it unlimited turns and 3 hours. It does not converge.

The infra casualties Running 12+ concurrent agent loops on a 16GB orchestrator box: swap at 100%, OOM kill, orphaned Strix children that survived the kill because the binary is a PyInstaller onefile (the inner app keeps running while the bootloader dies), sandbox containers holding 1.5-1.7G of writable layer each until I reaped them. At 6 parallel scans the inference engine's KV cache saturated at 83% and decode collapsed to 2-5 tok/s; the fix was an engine restart, because vLLM has no cache-clear endpoint. The gateway started 502ing on the giant re-sent contexts. None of that was the engine's fault — the scans just grow hundreds of thousands of tokens of conversation per target.

Where I land On this local model, Strix is a recon tool, not a vulnerability detector. It maps the surface, enumerates endpoints, and writes coverage reports, and it can't tell you which of those endpoints is actually exploitable. The numbers make it concrete: 5M+ tokens and up to 3 hours per deep target, zero completed scans with findings.

For real depth on internet-facing targets the honest cost is a frontier model at roughly $0.5-2 per scan, which I deliberately skipped because I wanted the local baseline first. This was the baseline. If you've run Strix (or similar agentic pentest tools) with a local model and gotten actual findings out of it, I'm curious what model and what target profile you used — happy to rerun the sweep with a different brain and post the diff.