8 functions fine, 11 hang: a debugging story about thresholds
"8 functions fine, 11 hang": a debugging story about thresholds
While wiring an embedded background worker into an app — a timer::every that drains a
jobs:: queue inside the web process — the server hung at startup. But only when the app also
had a use "file.lk" include. Remove the include, it started fine. Add it back, it froze.
The obvious hypothesis was a deadlock. It was wrong, and the thing that proved it wrong is worth
keeping: 8 include-functions bound fine; 11 hung. A deadlock doesn't care how many functions
you have — it hangs at a fixed point, every time. Something that's fine at 8 and dead at 11 isn't
a lock. It's growth. The question stopped being "where's the deadlock?" and became "what
scales super-linearly with the number of functions?"
The answer: registering the timer thread-clones the callback's closure environment. Every
function the included file defines shares that one environment, and a per-function guard was being
reset between siblings — so the shared environment was re-cloned once per function, each pass
re-visiting all the others. Classic O(N!) blow-up. Eight was instant; eleven was heat death.
The fix memoizes the cloned environments per pass; it also quietly corrected a semantic that was
subtly wrong and that nobody was looking for.
The durable takeaway is a diagnostic reflex, not a line of code: **when something hangs, the first
question is "is there a threshold?"** Constant delay is a deadlock; a wall you hit as input grows
is an explosion. That one question separates the two before you've read a single stack trace — and
it's saved us more than once since.