Reconcile and retain

Last updated

A Weald client holds a full local copy of every group it belongs to, and catching up means reconciling that copy.

Missing envelopes are found by range-based set reconciliation, documents converge as CRDTs with periodic snapshots, and history below a checkpoint can be deleted without breaking either.

Reconciliation

Two peers compare ranges of what they hold and never list everything. The exchange is bounded, so its cost does not grow with how long a client was away.

The relay assigns sequence numbers and gaps are legal, so reconciliation works over the set that exists. A gap does not indicate loss.

CRDT documents

Board state, tickets and plans are CRDT documents, so concurrent edits by two people converge. A semantic conflict, where two people decide different things, is shown to a human and never resolved silently.

A document writes a snapshot every 512 changes or 256 KB. A new device replays a snapshot and the changes after it. It never replays every change since the workspace began, which keeps a cold start affordable.

Cold start

A new device does not wait for the whole history before it is useful. Four phases run concurrently, with published targets.

What a new device has, and when
PhaseContentsTarget
SkeletonRoster, channels, board indexes, open ticketsUnder 5 seconds
RecentThe last 30 days of every groupUnder 60 seconds
BackfillOlder history, snapshots first, throttledBackground
MediaFetched on demandNever part of cold start

Backfill pauses below 20 percent battery. While backfill is incomplete, the client states in words the range it holds and shows no percentage. Once the index is complete it says nothing.

Search is client-side, over one encrypted index per workspace. The index is written in the same transaction that applies an envelope, so it cannot drift from the data. It is sealed at rest with a device-bound key that is never wrapped for anybody else, and it can be rebuilt from local state with no network.

There is no server-side search index, and there will not be one. A relay cannot index ciphertext. Giving it that ability would move the trust boundary this design exists to hold.

Retention and compaction

Compaction drops history below a checkpoint so a workspace stops growing forever. A client holding admin authority drives it, because the relay cannot read what it would be deciding about.

A workspace whose admin devices stay offline accumulates uncompacted history, unevicted agent leaves and a stale access set. Nothing on the server side repairs it. A weekly client health check covers that case, plus attestation liveness and access set freshness.

Specifications

The reconciliation algorithm, the channel model and the retention rules are published in the wire and conformance document sets.