Infra security review · for Muru & Sumanth
“We only expose 443, and mTLS 8080 for agents” turned out to be two-thirds right— there's a third port doing real work nobody accounted for.
01
Traefik — console, auth, minio, queen
ACME/Let's Encrypt TLS, host-based routing, Google-auth-gated where labeled. Working as intended.
engine — direct, bypassing Traefik
mTLS, client-cert verified at the engine itself. Correct to skip the shared proxy for this.
postgres — direct, bypassing Traefik
No mTLS, no proxy — password only, and the compose file's own fallback is postgres/postgres. Not part of the stated model at all.
02
Confirmed, not hypothetical
Mohan has directly observed live connections to 5432 from endpoint IPs. Tracing why: libs/metrics/postgres.go in schannel gives an agent a direct pgx connection to Postgres when its AgentConfig.Metrics.Backend is set to postgres — and that config is something the engine can push to any agent remotely, over the same 8080 mTLS stream agents already use for everything else.
No default/shipped config file in the repo sets this backend — but it's evidently active for at least some deployed agents right now. Agents run on arbitrary, unpredictable customer-network IPs, which is fundamentally incompatible with a curated source-IP allowlist: there's no list that represents “any agent, anywhere.”
03
Agents already have exactly one trusted network relationship: the mTLS gRPC connection to engine. Metrics should ride that same channel instead of a raw DB connection — the engine relays onward to queen (already a purpose-built message queue in this stack) or writes to Postgres itself as a single trusted service, the same way streams already does today.
This also fixes two problems the ACL alone can't: max_connections=200 would get exhausted fast if agents at scale ever really used the direct backend, and distributing DB credentials to every endpoint means one compromised agent leaks credentials usable against the whole database, not just its own data.
04
Once metrics no longer need a direct DB path, the only thing left justifying a second published port is the agent↔engine mTLS session on 8080 — and that doesn't need its own port either. A TLS ClientHello sends its SNI hostname unencrypted before the handshake completes, so Traefik can route purely on that, without terminating TLS, and hand the raw bytes straight to the engine's own mTLS listener.
engine.threatmatic.comon 443 — Traefik sees that SNI, passes the raw TCP stream straight through, untouched. The engine does its own mTLS handshake and client-cert check exactly as it does today on 8080 — nothing changes on the engine side except which port it's dialed on.
Real win beyond attack surface:agents on customer networks that only allow outbound 443 — common in locked-down corporate environments — would actually be able to reach the engine at all, which 8080 can't guarantee today.
SNI is one axis. There's a second one in the same handshake worth using deliberately, not leaving to default behavior — and a third capability, already inherent to gRPC, that turns the metrics problem in §03 from “route it differently” into “it was never a separate connection to begin with.”
Which hostname — routes to which backend
Sent in the clear before the handshake completes. engine.* vs. everything else — the passthrough decision in §04 above.
Which protocol the client wants — negotiated alongside SNI
A client advertises h2 for gRPC vs. http/1.1for plain HTTP in the same handshake. Traefik's TCP routers can match on this too, not just hostname — a second, independent axis if it's ever useful to split traffic on the same SNI by protocol rather than by service name.
Many logical streams — one physical connection
gRPC isHTTP/2. The agent already holds one persistent, already-mTLS'd connection to the engine for policy sync and commands. A metrics RPC is a new logical stream on that same connection, not a new port, connection, or handshake.
That last row is the actual point: fixing metrics isn't “give agents a different network path to Postgres” — it's recognizing agents never needed a second path at all. A ReportMetricsRPC added to the existing agent↔engine gRPC service rides the connection that's already open, already authenticated, already trusted. Zero new ports, zero new connections, zero new handshakes — just a new method on a service that already exists.
Two ways to terminate the mTLS itself — pick one:
Passthrough — Traefik never sees the handshake
What's described above. Engine keeps 100% ownership of client-cert validation, exactly as it does today. Minimal Traefik config; zero change to engine's trust model.
Traefik terminates mTLS itself
Router-level tls.options with clientAuth.clientAuthType: RequireAndVerifyClientCert, then HTTP/2 to the engine over the internal network. Centralizes cert policy (revocation, SAN rules) in one place if more services ever need mTLS — but engine stops validating certs itself, trusting Traefik instead. A real trust-model change, not just config.
05
Engine / agent side
Add a ReportMetrics RPC to the existing gRPC service, and retire libs/metrics/postgres.go's direct-connect option?
This is the actual fix for §02/§03 — not a new channel, a new method on the one that already exists. Removes the problem at the source instead of working around it.
Infra
Until that RPC ships, what covers the agents connecting today?
A pg_hba.conf source-IP ACL is scaffolded and ready (infra/db/pg_hba.conf), but needs the actual current agent IPs pulled from prod before it's safe to deploy — defense-in-depth once the RPC exists, not the fix on its own.
Infra
Passthrough (A) or Traefik-terminated mTLS (B) for engine.* on 443?
(A) is the smaller change and keeps engine's trust model exactly as-is today — the natural default unless there's already an appetite for centralizing cert policy across more than one service.