t4ngo@workstation:~/logs $ tail -f ./logs/case-studies

// SYSTEM LOGS

Case Studies

Each log is a real-world data problem, the plumbing that solved it, and the numbers on the other side.

SYS-LOG/0x21A
2025-04-18

Rebuilding the analytics warehouse from the wire up

// problem.txt
A 6-year-old warehouse with 400+ untested models, tribal SQL, and 8h refreshes was blocking every product decision.
// solution.txt
Introduced a staging/intermediate/marts layer, model contracts, and CI gates that block merges without tests and freshness checks.
// stack.txt
  • dbt
  • Snowflake
  • GitHub Actions
  • OpenLineage
// result.txt
  • ›Full refresh cut from 8h to 42m.
  • ›Test coverage 12% → 87% on critical models.
  • ›Incident MTTR down 63% quarter over quarter.
git://source →
SYS-LOG/0x22F
2025-06-02

Kafka to Iceberg without the 3AM pages

// problem.txt
Legacy Spark jobs were double-writing events and drifting from producer schemas — reconciliation ran nightly and often failed silently.
// solution.txt
Replaced the Spark hop with a Rust relay that owns schema evolution, idempotent commits to Iceberg, and per-topic backpressure.
// stack.txt
  • Kafka
  • Iceberg
  • Rust
  • Prometheus
// result.txt
  • ›Exactly-once semantics verified across 1.4B events.
  • ›Ingestion cost reduced by 58% on the same volume.
  • ›Zero silent schema drift incidents since launch.
git://source →
SYS-LOG/0x24B
2025-09-11

An Airflow platform that ops teams actually like

// problem.txt
Analytics engineers were building brittle DAGs on shared Airflow with no isolation, no lineage, and no local parity.
// solution.txt
Shipped a Terraform-defined Airflow on K8s with per-team namespaces, OpenLineage wiring, and a docker-compose dev twin.
// stack.txt
  • Airflow
  • Kubernetes
  • Terraform
  • OpenLineage
// result.txt
  • ›Onboarding for a new DAG author: 3 days → 2 hours.
  • ›Cross-team DAG collisions eliminated.
  • ›Lineage coverage: 100% of production tasks.
git://source →