Building a bulk QR generator that scales to millions of codes per job
18 July 2026
Generating one QR code is nothing. Generating a million in a single job — reliably, without timing out or blowing up the server's memory — is the whole ballgame. That's the problem bulkqr is built around. What follows is a high-level look at how the system's shaped. I've kept the internals out of it, but engineers should come away with a feel for how a job that big stays on its feet.
Why "just loop over the list" breaks
The obvious version — take the request, loop the rows, build every code, zip them, send them back — falls over at scale. Three reasons:
- Memory. A million rendered codes won't fit in one request's memory. You can't hold the whole result at once.
- Time. A request running for minutes will trip a timeout somewhere — browser, proxy, or server. Long requests are brittle.
- Failure. If a monolithic job dies at 90%, you've lost the lot, with nowhere sensible to pick up from.
Every choice below answers one of those three.
Stream, don't buffer
The key move: never hold the whole job in memory. Codes get built and passed straight on, so memory stays roughly flat whether the job is a thousand codes or a million. Output streams to storage as it's made, instead of piling up in one giant buffer at the end. Flat memory is what makes "how big is the job?" a question that stops mattering.
Accept fast, work on a queue
A big job shouldn't run inside the request that asks for it. The request checks the input, records the job, and returns quickly. The real work goes onto a queue, picked up separately. Now the user isn't holding a connection open for minutes, and a rush of big jobs piles up on the queue rather than knocking the system over.
Fan out to consumers
Whatever's on the queue gets handled by consumers running in parallel. Split a job into chunks and many consumers work through them at once — a job finishes in a fraction of the time one worker would take. Add consumers, throughput climbs. No single machine has to be big enough to do the lot on its own.
Keep every unit of work stateless
This is what makes the parallelism safe. Each chunk carries everything it needs to run on its own — no consumer leans on state held by another. Two payoffs:
- Any worker can take any chunk. No coordination, no sticky assignments.
- Retries are safe. A chunk that fails — a blip, a dead worker — just gets retried, because running it again gives the same result. Make the work idempotent and a retry is never a drama.
That's why a failure at 90% is a shrug rather than a disaster. You retry the handful of chunks that didn't finish, not the whole job.
Assemble from distributed storage
Finished codes get written to shared object storage as they're made, keyed so the job can find its pieces later. The final download — a ZIP, say — is stitched together by streaming those pieces on demand, not by gathering them all in one place first. Storage that scales sideways means the output size isn't capped by a single disk or process.
The themes underneath
Strip out the specifics and the same handful of ideas keep showing up:
- Bounded resource use, whatever the job size (streaming, chunking).
- Splitting the request from the work (queues).
- Keeping units of work independent (stateless, idempotent chunks).
- Scaling sideways — answer more load with more workers and more storage, not a bigger box.
None of it's clever for its own sake. It's the standard kit for high-throughput systems. The trick is sticking to it everywhere, so no part of the pipeline quietly assumes the whole job fits somewhere.
What it means for you
You never have to think about any of this. It's why 300 codes and 300,000 feel the same from where you sit: you submit, you get your codes. Small batches — up to 100 — don't even touch this machinery; they're built right in your browser. The heavy pipeline's there for when you need it.
Fancy stress-testing it? Start a job and see how it holds up.