Skip to content

Projects / Market (Ticker)

Market (Ticker)

A self-hosted, production-style market-data platform — one multiplexed feed, Redis-coordinated ingestion, TimescaleDB candles, computed indicators, and a live SSE dashboard.

Year
2026
Role
Solo
Focus
Backend · Infra
Status
Live
GoRedisTimescaleDBSSEDockerAWSTerraform
On this page
  1. Problem
  2. Approach
  3. Latency
  4. Results
  5. Lessons
  6. Go deeper

Problem

This portfolio needed to prove something a static site can't: that I can build and run a real distributed backend, not just talk about one. The constraint was specific — live-feeling data, sub-second updates, near-zero marginal cost, no user accounts, and honesty whenever the data behind it isn't a real market feed. A dashboard that quietly fakes numbers is worse than no dashboard at all.

Approach

  • A pluggable MarketSource interface decouples the pipeline from where ticks actually come from. v1 runs a simulated source (four regime symbols, five seeded cost-of-living cities); a real Finnhub adapter drops in later behind the same interface with no pipeline changes.
  • Ticks flow through a Redis Stream rather than an in-process channel, with a SET NX PX leader lease gating who's allowed to ingest. At v1's scale — one instance — the lease is non-load-bearing, but it's built for the horizontal-scale case on purpose, not retrofitted later.
  • An aggregator maintains OHLCV candles across five intervals at once (a configurable finest window, plus a fixed 5m/15m/1h/1d set), writing on every tick rather than on candle close so a server-rendered page always sees the live bar rather than a stale one.
  • Because a Redis Stream consumer group promises at-least-once delivery, processing is deduplicated by stream ID before anything accumulates, and candle writes are full-state upserts. High, low and close survive reprocessing on their own; volume is the one field that doesn't, which is exactly why the dedupe sits in front of it rather than relying on the upsert alone.
  • EMA and Wilder-smoothed RSI(14) are computed on read and cached in Redis, keyed per symbol, interval, and requested indicator set.
  • The browser never talks to the Go API directly. A single Next.js route proxies the SSE stream same-origin, so there's no public backend, no CORS, and no auth surface to manage.
  • On SIGTERM, the aggregator flushes every in-flight candle to Timescale and the ingestion loop releases its leader lease before the process exits, so a deploy hands over rather than dropping whatever was mid-flight.

Latency

Measured by the running system, not estimated: from the moment the source stamps a tick to the moment the hub writes it into a viewer's SSE stream. That span covers normalizing, the Redis Stream write and pub/sub, and the hub's fanout. The browser's own network hop and repaint aren't in it, because a server can't time what it never sees.

0.6 ms
tick → SSE write · p50 · 24h
2.5 ms
tick → SSE write · p99 · 24h
0.3 ticks/s
ingest · avg · 24h
4
peak live viewers · 24h
FinnhubIngest · GoRedisSSE hubBrowsertickcoalesce 1 s + Redis leaseXADD + PUBLISHasync · candle → TimescaleDBconsume + build eventevent: quote · SSEparse + paint · not measuredtick → SSE write · p50 0.6 ms · p99 2.5 ms · 24h

Read from Prometheus over the last 24 hours and refreshed every five minutes. Latency is timed once per delivery to each open viewer, so it only exists while someone is watching a live page. The source is the simulator until the real market adapter lands, so this measures this pipeline, not an upstream exchange.

Results

  • A live dashboard — candlestick chart with an EMA overlay, a real RSI(14) panel, a live markets table, and a cost-of-living series with expense-basket breakdowns — all driven by the same pipeline a real feed would use.
  • Runs alongside the rest of the site on one t4g.small box: Go, Redis, TimescaleDB, Next.js, and Caddy together, effectively $0 marginal infrastructure cost beyond the box that was already there.
  • Every figure carries the time it is dated to (asOf) end-to-end, from the Go API through to the page, and a feed that stops says so instead of falling back to invented numbers. The simulator exists for development and CI only.
  • Contract tests run against a real Ticker container in CI, so the frontend's assumptions about the API are checked against the API rather than against a fixture of it.

Lessons

  • The RSI Wilder-smoothing trap is real and well documented: a plain SMA-based RSI looks correct and quietly isn't. Validating against a hand-computed worked example — not just internal consistency — is what actually catches it.
  • Widening a primitive is cheaper than adding a parallel one. When the chart needed a full indicator series rather than a single latest value, the change touched seven files, including the registry contract and both indicator implementations. Nothing downstream broke, because the new shape contained the old one: the latest value became the last element of the series rather than a second thing to maintain.
  • Getting server rendering and a live stream right together is easy to get subtly wrong. Fetching live data from shared code can silently make an entire site render dynamically instead of statically — no error, no warning. It happened twice during the build, once feeding the command palette's symbol list and once the homepage's live-stat pill, and both fixes were the same shape: move the fetch into its own route, or behind an explicit revalidate window.
  • An at-least-once guarantee is something you implement, not something you're handed. A consumer group only redelivers if something claims entries that were read and never acknowledged, and the idempotent write everyone points at protects the write — not the accumulation sitting in front of it.

Go deeper