A routine nginx rebuild last week wiped evidence we didn't know we needed. The fix was two lines, but finding out we needed it took longer than it should have.
What we noticed
Pulling Google Analytics numbers to check on organic traffic, something looked off: roughly a quarter of the "Direct" channel's sessions were landing on /customer/login?returnTo=/payment/<plan>/<period> across nearly every pricing tier we sell, not two or three of them, almost all of them. Same browser fingerprint every time: unidentified browser, desktop, one specific country. Zero engagement on every single one.
That's not a customer browsing plans. That's a script walking every checkout URL it could find, almost certainly scraped straight off our own pricing page.
The evidence was already gone
We went to pull the nginx access log to find the source IP and got nothing. The container had been rebuilt a few days earlier for an unrelated fix, and access_log was pointed at a file inside the container instead of stdout. The rebuild wiped it clean. docker logs showed nothing, because there was nothing on stdout to show. The log pipeline we already had running, shipping container output to a central log store, had never seen a single line of it, because that pipeline only scrapes what containers write to stdout and stderr.
The bot's IP address is gone. There's no way to get it back now.
The fix
access_log /dev/stdout main;
error_log /dev/stderr warn;nginx treats /dev/stdout and /dev/stderr as ordinary files it can write to, so this isn't a special integration. The same log lines that used to sit in a file now go through the container's standard output, which our log-shipping setup was already reading from every other container we run.
The real lesson
The bot itself wasn't very interesting. Low-rate enumeration of public pricing URLs isn't much of a threat on its own. What mattered was that we had a monitoring stack that looked complete, with one container quietly opted out of it, because its base config still defaulted to writing log files the way it would have on a bare-metal box a decade ago. If that container gets rebuilt again next month, whatever it's doing then, we'll actually be able to see it.
If you're running nginx in Docker, it's worth checking your own access_log and error_log directives. A monitoring stack is only as complete as the containers that actually write to it.
