Run the vectors
Last updated
The Weald conformance corpus is declarative rather than a directory of opaque blobs. Each vector names the fields it builds an envelope from, the relay configuration it runs under, and the outcome it expects, so a reviewer can read what a test asserts without a decoder.
How a vector reads
A vector is data. It states the inputs, the configuration, and the expected result, so a reader can check that a test asserts the right thing.
The same envelope proves opposite outcomes on two configurations. That is how the corpus pins behaviour that depends on how a relay was started rather than on the bytes it received, and it is the part a hand-written test suite usually misses.
Documents
- Conformance vectors. The corpus an implementation is required to pass, as declarative data.
- Vector format. How to read a vector and what each expectation asserts.
- Threat model. The four boundaries, the attacker at each, and what is out of scope.
- Governance. Change classes, the breaking test, and the notice a change carries.
- Verification. The proofs a release has to reproduce, including the build digest.
- Database schema. What a relay stores on disk, which is what a backup has to capture.
Running them
Fetch the manifest, fetch the vectors it names, and run them against your implementation. Every file is served at a stable URL with cross-origin reads permitted, because a conformance run happens in somebody else's continuous integration.
The manifest carries a SHA-256 per file, so a corpus can be checked against the source it came from rather than trusted because of where it was downloaded.
What may change
Governance defines three change classes and a mechanical test for whether a change breaks an existing implementation. Breaking changes take a version, carry a thirty day comment period, and are announced before they ship. Versioning is the summary of that process.
The schema
The relay database schema is published, because what a relay stores on disk is what a backup has to capture and what a reviewer wants to grep. It is also the fastest way to satisfy yourself that there is no plaintext column anywhere in it.
Verify step 4 is the two-minute version of that check, against a running relay.