Three years in, remote for EliteITTeam (Manchester, UK) from Pakistan. The work I would point at first is a real-time GPU inference backend that went from a ~120-user ceiling on one L4 to a load-tested 500 on four L40Ses, where the finding that mattered was that the constraint had never been the hardware. The second is a capacity analysis where the correct recommendation turned out to be that we not build the system at all.
Three numbers, and where they came from
-
~120→500
concurrent users
Real-time GPU inference backend, one L4 to four L40Ses. A comfortable ceiling of 500 at 97.9% delivery, and 700 at 99.4% on the same four GPUs at a higher container density. GPU utilisation sat at 0–13% under normal load, reaching 83% only at 150 users: the ceiling was the Python GIL, not the hardware. How that was found →
-
61.9 s→1.9 s
Branching a 5 GiB Postgres database in pgoverlay, after finding that Postgres opens every file read-write before WAL replay and OverlayFS answers that with a full copy-up. The writable layer went from 5.05 GiB to 33 MiB. The diagnosis →
-
$206k/mo→$475–570k/mo
A correction, not a saving. The internal projection for scaling to 60,000 users assumed 200 users per GPU box; measured capacity was about 90, which more than doubles the real cost. I also found that GPUs are excluded from GCP's Flexible committed-use discounts, so the commitment plan could not rescue it either. My recommendation was that we not build it. We didn't.
Open source
-
pgoverlay
Copy-on-write Postgres branches on plain Docker or Kubernetes, built on OverlayFS: every branch shares one read-only base and stores only its own changes. Branching a 5 GiB database took 61.9 s and left a 5.05 GiB writable layer inside a system whose premise is that it copies nothing; it now takes 1.9 s and 33 MiB.
-
upgradescope
Kubernetes upgrade-readiness scanner. Reads a live cluster and answers whether it survives the next minor version in about a quarter of a second. Every claim in its API-lifecycle dataset carries a citation, and CI regenerates the dataset on every push and fails on any diff.
-
forgepoint
Takes a model from training through serving and back to retraining when it drifts, as a closed loop. The engineering problem is that eleven services spread across as many databases have to agree on what happened without a distributed transaction. That includes a cancelled workflow that still has to durably record that it was cancelled, on a context that is already dead. Go on Kubernetes, gRPC and NATS JetStream.
-
steward
Kubernetes operator. A 20-line Application CRD provisions Postgres, Redis and RabbitMQ, injects the credentials, and reports configuration drift. The hard part is telling an external edit apart from expected change, which it does by comparing spec generation against last-reconciled generation.
-
goqueue
Distributed message queue in Go: append-only log, leader election with no ZooKeeper or etcd, and a hierarchical timing wheel for O(1) delayed delivery. A cancel racing a bucket drain silently orphaned every remaining timer in that bucket for up to 7.76 days. No panic, no failing test, no log line.
-
sluice
L4/L7 proxy and load balancer with no dependencies in the request path. Its circuit breaker's half-open state admitted every caller (64 of 64 when 64 goroutines were released at once), so it was a label rather than admission control. 8,000 concurrent proxied connections measured; the ceiling was my test topology, not the proxy.
-
bookstore-kubernetes-guide
A 16-part, 115-chapter written guide taking one bookstore application from a container to production EKS: networking, security, GitOps, and the day-2 operations most guides stop before. Two runnable example trees, with the Terraform smoke-tested live against EKS. Published as a site, not a repo of notes.
Closed source
-
Media rendering & distribution pipeline
private
A data-driven system that turns a JSON scene graph into finished 1080p/4K video: TTS synthesis, word-level cue alignment, React/SVG component rendering, and an ffmpeg assemble-and-verify chain. A scheduler publishes across several third-party platforms on a calendar and reconciles local state against each platform's API, which always wins; drift escalates to a human. Running unattended on cron since June.
-
Real-time inference backendEliteITTeam
~120 on one L4 → 500 on four L40Ses under load test; 2,800 ms → 60 ms inference; 871 ms → 59 ms p50 once a
broadcast=Truefan-out was removed. Three of my own hypotheses were falsified along the way. -
Payments incident and data repairEliteITTeam
Root-caused a Stripe webhook failure and wrote the repair for the records it had corrupted: run dry-first, checksummed before and after, with a written rollback. Also took a reporting endpoint from 2.5 s to 650 ms along the way.
Writing
-
Postgres copied 5 GiB before recovery started
A copy-on-write branching benchmark that reported a full copy. OverlayFS copies a file up when it is opened read-write, and Postgres opens every file in the data directory read-write to fsync it before WAL replay. The durability pass therefore copied the whole dataset before a single query ran. Includes the measurement that exposed it and the limitation the fix does not remove.