Projects & case studies

Reporting queries ran against the same MySQL primary that served the application. As the dataset grew, a single analyst refreshing a dashboard could add noticeable latency to ordinary writes. The goal was to move analytical reads off the transactional database without introducing a nightly batch window.

The pipeline reads the MySQL binary log with Debezium, publishes change events to Kafka, and materialises them in ClickHouse. Deletes and updates are handled with a ReplacingMergeTree keyed on the primary key plus a monotonic version column, so the analytics tables converge on the same state as the source without requiring in-place mutations.

The interesting part was not the happy path but the operational envelope: schema changes, snapshot restarts, replication lag alerting, and deciding what "eventually consistent" is allowed to mean for each dashboard.

Teams frequently rebuild an image per environment, which quietly breaks the guarantee that what was tested is what ships. This pipeline builds a single image, tags it with the commit SHA, and promotes that same digest through staging and production.

Pull requests run lint, unit tests and a container build. Merges to the default branch push the image to a registry, run integration tests against it, and then deploy. Deployment jobs use GitHub environments so production requires an explicit approval and holds its own secrets.

Caching is deliberately conservative: dependency caches are keyed on lockfile hashes, and layer caching is scoped per branch so a poisoned cache cannot silently leak into main.

Adding read replicas is easy; routing to them safely is not. This work introduced an explicit routing layer plus a small set of rules about which reads are allowed to be stale.

Sessions that have just written are pinned to the primary for a bounded window, so a user never fails to see their own change. Everything else — listings, search, reports — is free to read from a replica. Lag is monitored and the router removes a replica from rotation when it falls behind a threshold.

The site you are reading. The frontend is fully static and ships almost no JavaScript; dynamic surfaces are the contact form and newsletter sign-up.

The backend is a FastAPI service running on a private server behind a Cloudflare Tunnel. It verifies Turnstile tokens, applies rate limiting, stores submissions in SQLite, and sends email via CyberPanel.

Storage and notification are separate states, so a notification failure never loses a message. Subscribers go through double opt-in — nothing is added to the list until a signed confirmation link is clicked.