Status — design, not yet built. This is a spec written in the project voice so it can be evaluated as a design. The correctness numbers below are the shape of the report the harness will produce (
Δ = 0is the target, not a measured result). It earns a place on the shelf only once the harness run backs it up.
The Problem
A usage event — customer X consumed Y units at time T — looks trivial until it meets production. The same event arrives twice because a client retried. An event shows up ten minutes late because a phone went through a tunnel. Events arrive out of order. A consumer crashes halfway through aggregating a window. Six weeks later a customer disputes an invoice and you have to prove the number.
Get any of these wrong and you either over-bill — chargebacks, churn, a support queue — or under-bill and quietly leak revenue, and the worst part is you can't tell which happened. Most pipelines "handle" this with an at-least-once queue and hope.
Usage-based billing is the most universal correctness problem in SaaS, and almost everyone gets late, duplicate, and out-of-order events subtly wrong.
The Approach
TallyMan ingests usage events at volume and turns them into billable rollups that stay exact under adversarial input. The design leans on a few decisions that do most of the work:
- Idempotency keys, not dedup-by-hope. Every event carries a client-supplied idempotency key; ingestion is exactly-once at the effect level even on at-least-once delivery. A retry is a no-op, not a double charge.
- Event-time windows with a watermark. Usage is aggregated by when it happened, not when it arrived. A bounded lateness window admits stragglers; a watermark decides when a period is safe to close.
- Late events reconcile, they don't corrupt. An event that arrives past its window lands in a reconciliation ledger as an explicit adjustment against a closed period — never a silent mutation of a number a customer has already seen.
- Immutable log → derived rollups. The raw event log is the source of truth; billable rollups are a materialized view you can recompute and diff. That recompute-and-diff is the answer to the six-week-late dispute.
- Closed periods are append-only. Once a billing period is finalized, corrections are adjustment entries, never in-place edits — auditability by construction.
Staying exact when events don't cooperate
The interesting engineering is in the failure modes, not the happy path — the same discipline the DropBear merge and idem.sh idempotent verbs are built on, pointed at money instead of files.
- A duplicate is absorbed, not counted. The idempotency key is the unit of dedup; delivery can be at-least-once because the effect is exactly-once.
- A missing event is never read as zero. Absence means "not yet," the same way DropBear reads a missing root as offline rather than deleted — a stalled consumer never bills a customer for silence.
- A crash mid-window reconciles on restart. Aggregation is recovered from the immutable log, not from in-memory partials, so a process that dies between two events resumes without double-counting or losing a unit.
- Backpressure to the source. Straight from LogBus: a slow downstream — the warehouse, the invoice service — stalls ingestion rather than dropping billable events or growing memory without bound.
The proof — a correctness report
The design is only worth anything if it's shown, so the centerpiece is an adversarial harness that injects each failure mode into the same event stream and checks the billed total against ground truth. The target is a single column of zeros.
| Scenario | Events | Duplicates | Late (> window) | Out-of-order | Billed | Truth | Δ |
|---|---|---|---|---|---|---|---|
| Clean | 1,000,000 | 0 | 0 | 0 | — | — | — |
| 20% client retries | 1,000,000 | 200,000 | 0 | 0 | — | — | — |
| Late arrivals | 1,000,000 | 0 | 50,000 | 0 | — | — | — |
| Out-of-order | 1,000,000 | 0 | 0 | 100,000 | — | — | — |
| Consumer crash mid-window | 1,000,000 | — | — | — | — | — | — |
| Adversarial (all at once) | 1,000,000 | 200,000 | 50,000 | 100,000 | — | — | — |
| Invariant disabled (control) | 1,000,000 | 200,000 | 0 | 0 | — | — | ≠ 0 |
The last row is the SearchLab move: deliberately switch off the idempotency check and show the total drift, because a result you can only trust by measuring should be shown drifting when the guard is gone. A throughput/latency row (events/s, p99 ingest) rides alongside so the same page answers "is it exact" and "does it scale."
Where it stands
A design, not a running system — written up here so the shape can be judged before it's built. The plan is to ship the exactness proof and the throughput numbers first on a single node (Postgres as both the append-only log and the rollup store), and leave real Kafka ingestion, multi-node aggregation, and any UI as stated gaps. The correctness report alone is the deliverable; a half-built "platform" is not. Promote off design status — and onto the featured shelf — only once the Δ = 0 column is real.