High-throughput telemetry ingestion at 1,000+ RPS.
The project started with a crashed BTS live stream on Weverse. Rather than chase the symptom, I engineered a production-grade, horizontally scalable real-time streaming pipeline to simulate global live-broadcast telemetry — ingest, buffer, aggregate, and persist high-concurrency events under intense traffic spikes without turning the database into a bottleneck.
Streaming raw telemetry directly into a relational DB under millions of concurrent viewers causes connection pool saturation, lock contention on shared indices, backpressure-driven silent packet drops, and runaway query latency — the classic write-storm failure mode.
The database stopped being the bottleneck. Throughput scales horizontally with gateway pods, tail latency stays flat under 1,000+ RPS, and the same code deploys unchanged onto GKE + Pub/Sub + Dataflow + Cloud SQL.
Core philosophy: clean algorithmic code is the foundation; cloud is just the tool. Built a local cloud-native prototype that solves the scale problem in software first — so scaling out later is cheap.
Decoupled write path: async FastAPI gateway → thread-safe TCP socket queue (shock absorber) → stateful in-memory compute worker with 10-second fixed tumbling windows.
Compressed 6,000+ distinct DB operations per window into a single batched ON CONFLICT DO UPDATE upsert, driving index lock contention and query latency toward zero.
Every component maps 1:1 to a GCP service for enterprise readiness: FastAPI gateway → Cloud Run, TCP buffer → Pub/Sub, stateful worker → Dataflow (Apache Beam), Postgres → Cloud SQL, dashboard → Looker Studio, load sim → Cloud Tasks.
Validated with Locust distributed swarm load testing to triage bottlenecks, analyze backpressure, and confirm pipeline fault tolerance under heavy network loads.
"The lesson was that scale is a software-architecture problem before it's an infrastructure problem — the cloud doesn't fix a bad design, it just makes it expensive."