specdec-findings: Modules

Modules: standalone copies of the record's tools

8 parts of the record's code and method that stand on their own, one catalogue card each, beside the record's notes on what it does, how to use it and what to know first. The packaged folders are not published as downloads; instead each card carries one command that builds its folder from the repository at tag v1.0.0, checking every file against its full sha256 before it writes any. The file list under each card links every file to the same file at that tag, its sha256 prefix checked against the tag when this page's data was extracted (65 files). The card text comes from the module catalogue kept with the private record, so it carries no source chips; a test line gives the counts from running that copy's tests.

exact verifier

Exact verifier: the speculative-decoding accept/reject step in numpy

sideband · folder

6 files, 29146 bytes; MIT licence.

Contents: spec_step; run_block; residual_distribution; analytic_acceptance; StepOutcome; BlockOutcome; RunMetrics.

Tests of this copy: 24 passed

How to use this tool

A speculative decoder lets a small draft model guess the next tokens and a large model check them. This is the checking rule: it keeps or replaces each guess so that the text that comes out follows the large model's own probabilities exactly, whatever the draft guessed. The tests prove that by counting through every outcome.

Copy the command under Get this folder, run it, then run the tests to see the rule hold.
In your code, import spec_step from sideband.specdec. Give it a numpy random generator, then the large model's probabilities, then the draft's, for one position over the same vocabulary.
It returns a StepOutcome: the token the draft proposed, whether it was kept, and the token that came out.
For a block of several guesses use run_block, with one more large model distribution than draft distributions; the extra one gives a bonus token when every guess is kept.
If you write a faster version, make these tests pass against it before you trust it.

A numpy implementation of the exact speculative-decoding accept/reject step (Leviathan et al. 2023) with tests that check by enumeration that the emitted token follows the target distribution for any draft distribution; for researchers building or changing speculative decoders.

Get this folder

One command makes a new folder named exact-verifier inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 6 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need numpy and pytest: pip install numpy pytest, then cd exact-verifier and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('exact-verifier')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('src/sideband/__init__.py', 'sideband/src/sideband/__init__.py', 'b3123882e5c63944e9ecadcf31126ddbd513be73b9829ae89ee095dca685946f'),
    ('src/sideband/dists.py', 'sideband/src/sideband/dists.py', 'a8fb156eff2141061fa7b7d1ae61de364797aebb77acf7a7425f25fb7423e280'),
    ('src/sideband/specdec.py', 'sideband/src/sideband/specdec.py', '80502d0f5695a56f650aa1d166afaa7b34683628480bed703b8fd34f4640c65d'),
    ('src/sideband/steering.py', 'sideband/src/sideband/steering.py', '3271311b2303ba317036f01ca581a16da00bb22598eefd65e65d50a6b55ffff7'),
    ('tests/test_specdec.py', 'sideband/tests/test_specdec.py', '4791d46cc47cdf1c3d55623e7d6b203908375fff7fbdc136112c0348aff63ca2'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests', 'pythonpath = src', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

In the record this file is the single implementation of accept, reject and residual (DECISIONS.md D-3: the real-model adapter converts logits to numpy and calls spec_step, and anyone re-implementing the verifier for speed must first make these numpy tests pass against their implementation). Its docstring states the property the tests check by enumeration: for any draft distribution, steered or not, the emitted token is distributed exactly as the target distribution, so steering can only change speed, never output quality.

Use

Copy src/sideband/ into your project, or vendor specdec.py and dists.py together (specdec imports normalize, sample and tv_distance from dists).
steering.py is here only because the test file uses its bucket_bias; it is not needed to call spec_step.
Run python -m pytest from this folder: pytest.ini points it at the copied tests and this folder's src/.

Good to know

Per-step acceptance is analytically 1 - TV(p, q), and that identity is tested too.
The docstring names the record's real-model adapter (sideband.real.hf_specdec, not copied) and the test file, which is copied here.

The 6 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
src/sideband/__init__.py	sideband/src/sideband/__init__.py	b3123882e5c6	87
src/sideband/dists.py	sideband/src/sideband/dists.py	a8fb156eff21	2115
src/sideband/specdec.py	sideband/src/sideband/specdec.py	80502d0f5695	7210
src/sideband/steering.py	sideband/src/sideband/steering.py	3271311b2303	6217
tests/test_specdec.py	sideband/tests/test_specdec.py	4791d46cc47c	12447

cluster bootstrap stats

Cluster bootstrap: confidence intervals for prompt-clustered evaluations

sideband · folder

7 files, 35410 bytes; MIT licence.

Contents: clopper_pearson; cluster_bootstrap_ci; paired_cluster_delta_ci; paired_ratio_delta_ci; paired_ratio_delta_of_deltas_ci; e6_gate.pair; e6_gate.evaluate.

Tests of this copy: 38 passed, 2 skipped

How to use this tool

When two methods are scored on the same prompts, results from one prompt tend to move together, so treating every result as independent makes the uncertainty look smaller than it is. These functions give confidence intervals that resample whole prompts, and when comparing two methods they resample both with the same draw, so the difference is a fair paired one.

Copy the command under Get this folder, run it, then run the tests.
For each prompt, count the successes of method A, the successes of method B and the number of trials, with the prompts in the same order in all three arrays.
Call paired_cluster_delta_ci with those three arrays. It returns the difference between the two success rates and the low and high ends of its 95% interval.
For one method alone use cluster_bootstrap_ci. With nothing to group by, clopper_pearson gives the exact interval for k successes in n trials.
The same seed always gives the same interval, so fix the seed before you look at results.

Numpy-only confidence intervals for clustered evaluation data: an exact Clopper-Pearson interval, a prompt-cluster percentile bootstrap, and paired delta and paired ratio-delta bootstraps that apply one resampling draw to both arms; for researchers comparing two methods on the same prompts.

Get this folder

One command makes a new folder named cluster-bootstrap-stats inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 7 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need numpy and pytest: pip install numpy pytest, then cd cluster-bootstrap-stats and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('cluster-bootstrap-stats')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('src/sideband/__init__.py', 'sideband/src/sideband/__init__.py', 'b3123882e5c63944e9ecadcf31126ddbd513be73b9829ae89ee095dca685946f'),
    ('src/sideband/dists.py', 'sideband/src/sideband/dists.py', 'a8fb156eff2141061fa7b7d1ae61de364797aebb77acf7a7425f25fb7423e280'),
    ('src/sideband/stats.py', 'sideband/src/sideband/stats.py', '2def9e60b56a555740a98015ef44a73866712de002787fa277ac2042a880d4cd'),
    ('tests/test_e6_gate.py', 'sideband/tests/test_e6_gate.py', 'ce11bb5c19a06e91bee6d281b3f2db0cb4f16a75bc3db3d7e06c1749999e3a20'),
    ('tests/test_stats.py', 'sideband/tests/test_stats.py', 'a8b71dd3b498b35116235da824df6f79e4fd885af1e30fb7e858bdead2e9613c'),
    ('tools/e6_gate.py', 'sideband/tools/e6_gate.py', 'ca1f5e7ae7ce77f8fb44268098a8a762cf9599d5b56f691ae6703db5e93f79d5'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests', 'pythonpath = src', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

stats.py is the estimator the record's root README recomputes a finding with in its worked example (paired_ratio_delta_ci over a committed state file), and the one its conversion gate, tools/e6_gate.py, imports; several record instruments carry bootstraps of their own beside it. Its docstring reports the pair of estimators together on purpose: the exact binomial interval assumes independent trials, which positions within a prompt are not, and the percentile bootstrap under-covers at small cluster counts (its docstring gives the measured coverage at 10, 30 and 60 clusters for a nominal 0.95), so neither end is conservative and both are reported.

Use

stats.py needs only the Array alias from dists.py; copy both, or inline the alias.
tools/e6_gate.py is a worked example of the paired-delta gate, not an API: its population table, results path and bar constants are the record's own runs.
Run python -m pytest from this folder. Two legs of test_e6_gate skip here because they read the record's committed verdict summaries.

Good to know

The Clopper-Pearson reference values in the tests were generated once from scipy.stats.beta and committed; scipy is not a dependency.
The bootstrap is deterministic given its seed; the paired forms apply one resampling draw to both arms, which is what makes the delta a paired delta.

The 7 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
src/sideband/__init__.py	sideband/src/sideband/__init__.py	b3123882e5c6	87
src/sideband/dists.py	sideband/src/sideband/dists.py	a8fb156eff21	2115
src/sideband/stats.py	sideband/src/sideband/stats.py	2def9e60b56a	11457
tests/test_e6_gate.py	sideband/tests/test_e6_gate.py	ce11bb5c19a0	4057
tests/test_stats.py	sideband/tests/test_stats.py	a8b71dd3b498	8877
tools/e6_gate.py	sideband/tools/e6_gate.py	ca1f5e7ae7ce	7747

rank census

Rank census: does a model's token ranking reproduce across environments?

amber · folder

4 files, 17769 bytes; MIT licence.

Contents: rank_census; rank_census_topk; calibrate_tau; calibrate_tau_topk; RankCensus; stable_topk; top2_gap; canonical_topk; aligned_delta.

Tests of this copy: 10 passed

How to use this tool

The same model on two machines, or at two batch sizes, can rank its candidate next tokens a little differently. This tool takes the scores from both runs at the same positions and reports how often the top token, the top few tokens and their order change, and how close the top two scores were where the top token changed.

Copy the command under Get this folder, run it, then run the tests.
Save the model's scores (logits) at the same positions from each run as a numpy array with one row per position and one column per vocabulary entry.
Call rank_census with the reference run first and the other run second.
Read the RankCensus it returns: how often the top token flipped, how often the top 1, 8 and 64 changed as a set and in order (the ks argument sets those sizes), and the smallest, median and largest gap between the top two scores where the top token flipped.
Its only dependency is numpy, so you can copy rankcensus.py on its own.

A numpy instrument that compares two logit dumps of the same positions (for example two devices or batch sizes) and reports top-1, top-k set and top-k order flip rates, the top-2 gap at each flip, a calibrated tolerance and flip burstiness; for researchers checking whether a model's token ranking reproduces across environments.

Get this folder

One command makes a new folder named rank-census inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 4 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need numpy and pytest: pip install numpy pytest, then cd rank-census and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('rank-census')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('src/amber/__init__.py', 'amber/src/amber/__init__.py', 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'),
    ('src/amber/rankcensus.py', 'amber/src/amber/rankcensus.py', 'ede73b25d417ec7187c7bedb0a556587ffbba333825f61bf9d0deb745638724c'),
    ('tests/test_rankcensus.py', 'amber/tests/test_rankcensus.py', '8693ab4f8634e297cbe614bce286210d31d7327ab4a3df10b5f740a1aeee14cb'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests', 'pythonpath = src', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

In amber this is the instrument for its rank-stability question: amber's determinism contract is that the draft's token ordering reproduces at decode time, not its logits bitwise, and this module measures how far two logit dumps of the same teacher-forced positions disagree in exactly that quantity. Its docstring explains why every top-1 flip carries the reference top-2 gap (a flip at a hairline gap is floating-point noise, a flip at a wide gap is a bug) and why the tolerance is calibrated from a benign perturbation rather than picked.

Use

One file with no internal imports: copy src/amber/rankcensus.py alone, or the folder.
Run python -m pytest from this folder.

Good to know

The docstring credits the owner's other research projects by name as the sources of its rules (the calibrated tolerance, the near-tie budget, the drift-census shape); those are provenance notes, not imports, and would be reworded for a standalone card.
Order convention is descending value, ascending index on ties, the same as amber's codec, so the instrument and the codec agree on what a rank is.

The 4 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
src/amber/__init__.py	amber/src/amber/__init__.py	e3b0c44298fc	0
src/amber/rankcensus.py	amber/src/amber/rankcensus.py	ede73b25d417	11471
tests/test_rankcensus.py	amber/tests/test_rankcensus.py	8693ab4f8634	5228

synthetic testbed

Synthetic testbed: a CPU world for draft-steering experiments

sideband · folder

14 files, 78787 bytes; MIT licence.

Contents: WorldConfig; World; TelemetryDataset; run_simulation; train_sidecar; SimResult; POLICIES; HashSketch; SoftmaxRegression; GatedSteerer; posterior_bias; bucket_bias.

Tests of this copy: 68 passed

How to use this tool

A small made up world that runs on an ordinary CPU: a large model, a draft model and a stream of side information, all generated from a hidden topic that changes now and then. Because the hidden truth is known, you can try an idea for steering the draft, and check the code behind it, before spending time on real models.

Copy the command under Get this folder, run it, then run the tests.
Make a world with World(WorldConfig()); change the fields of WorldConfig, its seed among them, to shape it.
Call run_simulation(world, seed, n_positions, gamma, policy=...) once for each name in POLICIES and compare the SimResult each returns; gamma is the number of draft guesses per check.
The learned policy needs a trained model first, which train_sidecar makes.
Know its limit: here the side information and the large model come from the same hidden topic, so no steering can beat the free baseline in this world. Whether real models allow more is a question it cannot answer.

A numpy synthetic world (a sticky concept chain with target, draft and hashed telemetry) wired to an exact speculative decoder and five policies (vanilla, two oracles, the free-information baseline and a learned sidecar), so steering and analysis code can be checked against known ground truth on CPU; for researchers prototyping draft-steering ideas.

Get this folder

One command makes a new folder named synthetic-testbed inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 14 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need numpy and pytest: pip install numpy pytest, then cd synthetic-testbed and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('synthetic-testbed')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('src/sideband/__init__.py', 'sideband/src/sideband/__init__.py', 'b3123882e5c63944e9ecadcf31126ddbd513be73b9829ae89ee095dca685946f'),
    ('src/sideband/buckets.py', 'sideband/src/sideband/buckets.py', '6a67905cfed5cd15dc0caffbfd8953f5d206cdc4b3291e6093bbbf00bf1b60b9'),
    ('src/sideband/dists.py', 'sideband/src/sideband/dists.py', 'a8fb156eff2141061fa7b7d1ae61de364797aebb77acf7a7425f25fb7423e280'),
    ('src/sideband/pipeline.py', 'sideband/src/sideband/pipeline.py', '8262c73cbcf67105aa1ea416f453faa09bf7d34c4fc75b27fb365a348ad5115b'),
    ('src/sideband/sidecar.py', 'sideband/src/sideband/sidecar.py', '50b5c5ed452759365def8084d2cd36629d60f3b19d0c3a649de68e8f133363af'),
    ('src/sideband/sketch.py', 'sideband/src/sideband/sketch.py', '276b46e4ee26e175c7becc6bd7b3721e409b19392d9ed6b55c7a64c8ec228295'),
    ('src/sideband/specdec.py', 'sideband/src/sideband/specdec.py', '80502d0f5695a56f650aa1d166afaa7b34683628480bed703b8fd34f4640c65d'),
    ('src/sideband/steering.py', 'sideband/src/sideband/steering.py', '3271311b2303ba317036f01ca581a16da00bb22598eefd65e65d50a6b55ffff7'),
    ('src/sideband/synthetic.py', 'sideband/src/sideband/synthetic.py', '388b184ccbf08ac3cad7818dc309d546c3b9d3bf41e87c074140b0ff6561f148'),
    ('tests/test_posterior_mechanism.py', 'sideband/tests/test_posterior_mechanism.py', '5a3c10c736b9325e81c476fbd6db15d973657f7c334573c65abfd80161ea6ddd'),
    ('tests/test_sketch_buckets.py', 'sideband/tests/test_sketch_buckets.py', '5e7996566a7c3252bcd474090f5b9ee11b00cac7f5ecbbf8667fff300db3d5b2'),
    ('tests/test_steering_pipeline.py', 'sideband/tests/test_steering_pipeline.py', '09d16f839058cb537e62dd4f882fbccd261cdb547af9cbfba3c0880e22effc70'),
    ('tests/test_synthetic_sidecar.py', 'sideband/tests/test_synthetic_sidecar.py', '215b0100cd0e5f843ba7d37781a80b6ae5ce8bb7bf01c4ec61e9657a20aa526e'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests', 'pythonpath = src', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

The record's findings index says its first arc built the verifier and this testbed and used it to fix the shapes that every later experiment inherits (docs/FINDINGS_INDEX.md, the Synthetic world arc), and two of its decisions live in pipeline.py: the free-information baseline policy (DECISIONS.md D-5: any telemetry-steering claim is compared against steering from the target's own distribution at the last verified position, which every speculative decoder computes anyway) and the isolated telemetry random stream (D-6: policies are compared at identical seeds so trajectories are paired, and telemetry draws from its own generator so a policy that reads it cannot perturb the shared draft and verify trajectory).

Use

Copy src/sideband/ whole: pipeline.py imports synthetic, sidecar, steering, specdec and dists, and synthetic.py imports sketch.
buckets.py is here to keep test_sketch_buckets.py whole; synthetic.py does not import it.
Run python -m pytest from this folder.

Good to know

Regime warning from synthetic.py's own docstring: telemetry and the target's next-token distribution are functions of the same latent concept here, so a telemetry sidecar can at best match the free-information baseline, and whether real hidden-state sketches carry more is a question this world cannot answer by construction.
The copied files cite the record's decision, finding and open-question ids (D-3, D-5, D-6, D-7, D-8, D-9, D-10, D-11, F-3, F-7, F-8, OQ-1, OQ-4); those read against the record's DECISIONS.md, FINDINGS.md and OPEN_QUESTIONS.md.

The 14 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
src/sideband/__init__.py	sideband/src/sideband/__init__.py	b3123882e5c6	87
src/sideband/buckets.py	sideband/src/sideband/buckets.py	6a67905cfed5	4702
src/sideband/dists.py	sideband/src/sideband/dists.py	a8fb156eff21	2115
src/sideband/pipeline.py	sideband/src/sideband/pipeline.py	8262c73cbcf6	10907
src/sideband/sidecar.py	sideband/src/sideband/sidecar.py	50b5c5ed4527	3476
src/sideband/sketch.py	sideband/src/sideband/sketch.py	276b46e4ee26	2435
src/sideband/specdec.py	sideband/src/sideband/specdec.py	80502d0f5695	7210
src/sideband/steering.py	sideband/src/sideband/steering.py	3271311b2303	6217
src/sideband/synthetic.py	sideband/src/sideband/synthetic.py	388b184ccbf0	8735
tests/test_posterior_mechanism.py	sideband/tests/test_posterior_mechanism.py	5a3c10c736b9	5825
tests/test_sketch_buckets.py	sideband/tests/test_sketch_buckets.py	5e7996566a7c	5605
tests/test_steering_pipeline.py	sideband/tests/test_steering_pipeline.py	09d16f839058	15594
tests/test_synthetic_sidecar.py	sideband/tests/test_synthetic_sidecar.py	215b0100cd0e	4809

findings lint

Findings lint: no regime line and no tracked source, no finding

sideband · folder

3 files, 17354 bytes; MIT licence.

Contents: main; lint_text; parse_entries; source_paths; source_lines.

Tests of this copy: 1 failed, 4 passed

How to use this tool

A check for a research log. Every finding in FINDINGS.md must say under what conditions it was measured (a regime line) and point to evidence files that git tracks (source lines). The check fails when a finding is missing either, or cites a file that does not exist or that git does not track.

Copy the command under Get this folder and run it, then copy lint_findings.py from the new folder's tools folder into a tools folder at the root of your repository. It finds the root from where it sits and reads FINDINGS.md there.
Give each finding a heading that starts ## F-, a line that starts - regime: and one or more lines that start - source: naming files under results/, tests/, docs/ or the other folders listed at the top of the file.
Run python tools/lint_findings.py. It prints FAIL and exits with code 1 when a finding fails; add it to your make lint target and to your CI so it runs every time.
Change the list of evidence folders at the top of the file to fit your layout.
In this folder one of its five tests fails as expected: it reads the record's own FINDINGS.md, which is not copied.

A stdlib-only lint that fails a FINDINGS.md whose entries lack a regime line or a source line pointing at paths git tracks; for research repositories kept by people or coding agents, wired to make lint and CI.

Get this folder

One command makes a new folder named findings-lint inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 3 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need pytest: pip install pytest, then cd findings-lint and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('findings-lint')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('tests/test_lint_findings.py', 'sideband/tests/test_lint_findings.py', '797bab9c41eb1c9fb6cdf08f0318f627b271b7f987213957bb25aa70c458258f'),
    ('tools/lint_findings.py', 'sideband/tools/lint_findings.py', '5ed53a4ca06a58200ed60a2e20fd55775afa042a890e27e25bd0b1acc1bb622e'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

The record's decision D-4 (findings carry regime and evidence, lint-enforced; its CLAUDE.md puts it as 'no regime, no finding') says every FINDINGS entry must have a regime line and a source line with existing evidence paths, and this lint is what enforces it on every make lint and CI run. METHOD.md's section on reusing the method in another repository names this file first of its three-file kit.

Use

Copy tools/lint_findings.py into a tools/ folder under your repository root: it finds the root from its own location and reads FINDINGS.md there. Run python tools/lint_findings.py; wire it to make lint and CI.
The prefixes it checks source paths under (results/, experiments/, tests/, docs/, src/, data/, tools/) are a constant at the top of the file; change them for your layout.
Run python -m pytest from this folder: one of the five tests reads the record's own FINDINGS.md, which is not in this folder, and fails with a file-not-found error; the other four pass on fixtures.

Good to know

amber carries a different, shorter tool under the same file name; this is sideband's.
The lint checks that cited paths exist and that git tracks them; when git cannot answer it falls back to the existence check and says so in its OK line, except under CI (the CI environment variable), where that is a failure.

The 3 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
tests/test_lint_findings.py	sideband/tests/test_lint_findings.py	797bab9c41eb	5499
tools/lint_findings.py	sideband/tools/lint_findings.py	5ed53a4ca06a	10785

landing text

Landing text: generated prose whose every number traces to a payload field

sideband · folder

3 files, 52561 bytes; MIT licence.

Contents: Pair; Region; Doc; fill_template; score; assert_banned; unknown_numbers; check_texts; main.

Tests of this copy: 22 passed, 1 skipped

How to use this tool

A helper for the text that reports a result. Instead of typing numbers into a sentence, you pass each one through the tool, which records where it came from; a check then flags any number in the finished text that no result file holds. It also blocks banned phrases and reports mixed line endings.

Copy the command under Get this folder, run it, then run the tests.
In the script that writes your text, load your primary and replication result files with Pair and insert every number through it. Pair expects this record's result layout, so adapt its field names to yours first.
Call Pair.write with a folder and your texts; it saves them beside a numbers.json file listing every number used.
Run python tools/landing_text.py check numbers.json --files your_text.md. Each number it cannot trace is printed as a FINDING and the exit code is 1.
python tools/landing_text.py eol reports the line ending of each file.

A stdlib-only tool for writing the text that lands a result: pairs of generated sentences and the payload fields they quote, a check that no number in the text is unknown to a payload, a banned-phrase assertion, a line-ending normaliser and a CLI; for research repositories whose results are landed by a coding agent.

Get this folder

One command makes a new folder named landing-text inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 3 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need pytest: pip install pytest, then cd landing-text and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('landing-text')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('tests/test_landing_text.py', 'sideband/tests/test_landing_text.py', '6db101c10b227c2e3114c86a79738067a2d72f43ac68327454224b135c487881'),
    ('tools/landing_text.py', 'sideband/tools/landing_text.py', '7bddcc8774e624d53618a029d67bd8e033ed18973110552d05f8b849e158c7aa'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

The record keeps the rule that numbers in prose come only from files read in that session (its CLAUDE.md working practices and docs/METHOD.md), and docs/CONVENTIONS.md records the generated-landing practice this tool served in the later lanes: every number in a generated sentence must trace to a registered payload field, and the text refuses banned phrases before it is written.

Use

Copy tools/landing_text.py; its Pair expects the record's payload schema (conclusion.labels, conclusion.deltas, provenance), so adapt the field names to your payloads.
Run python -m pytest from this folder. One test needs the record's committed R2 payloads and skips without them.

Good to know

This tool and the findings lint were catalogued as one registration kit together with an evidence-pin tool; the pin tool is bound to the record's own digest file and is not copied here.

The 3 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
tests/test_landing_text.py	sideband/tests/test_landing_text.py	6db101c10b22	21823
tools/landing_text.py	sideband/tools/landing_text.py	7bddcc8774e6	29668

critic lane

Critic lane: an adversarial reviewer agent that recomputes from state files

sideband · agent

3 files, 10201 bytes; MIT licence.

Contents: agent sideband-critic (modes: recompute, diff); skill /critic.

Tests of this copy: no tests: a prompt, not code

How to use this tool

Instructions that make Claude Code a second, sceptical reviewer for your experiments. Before you commit a result, it checks the change against a nine item checklist, marks each objection as blocking or advisory, and can recompute the claimed numbers from your saved run files instead of trusting the text.

Copy the command under Get this folder and run it; the folder holds the two instruction files and the licence.
Put sideband-critic.md in your project's .claude/agents/ folder and SKILL.md at .claude/skills/critic/SKILL.md.
Rewrite the nine checklist items for your own work: as copied they refer to this record's decisions and findings.
In Claude Code, type /critic, or ask it to run the critic on a change, a finding or a result file. Ask for recompute mode when you want the numbers checked.
The agent file names a model and the tools it may use; set both for your own setup.

A Claude Code review-agent prompt and a /critic skill that check an experiment change or a finding against a nine-item checklist, tag each objection BLOCKING or ADVISORY, and in recompute mode re-derive the asserted numbers from committed state files; for researchers who run experiments with a coding agent.

Get this folder

One command makes a new folder named critic-lane inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 3 files listed below from the repository at tag v1.0.0, checks each against its full sha256 and writes nothing unless every one matches and the folder is new. The use notes say where each file goes from there.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('critic-lane')
files = [
    ('LICENSE', 'LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('sideband-critic.md', 'sideband/.claude/agents/sideband-critic.md', '54b640ee2d3a46dd9ff12d4bac85b2caa7a9006e4d9cdc8e4c761293bdbddc34'),
    ('skills/critic/SKILL.md', 'sideband/.claude/skills/critic/SKILL.md', '182effe690b94f6b4228812bdbb602d0770e8f4f9986f48350c25ee7f5070993'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

The record's workflow runs this agent before any finding is committed and before experiment changes merge; its highest-value mode recomputes the asserted labels and numbers from the committed state files rather than reading the diff. METHOD.md's reuse section names it second of its three-file kit and says to rewrite the nine checklist items for your domain while keeping the shape: named leak channels, the dominated baseline, regime abuse, survivorship, pairing, the banned claim class, kill-criteria dodging, the BLOCKING and ADVISORY tags, and the closing 'What would change my mind:'.

Use

Save sideband-critic.md as .claude/agents/sideband-critic.md in your project and skills/critic/SKILL.md as .claude/skills/critic/SKILL.md; then ask Claude Code to run the critic on a diff, a finding or a payload.
Rewrite the nine checklist items for your domain before relying on it: as copied they cite this record's decisions, findings and a kill criterion (D-2, D-5, D-6, D-7, D-9, F-3, F-5, F-6, K-5).
The agent file pins model: opus and a tool allowlist (the read tools plus python and git commands; the file itself says nothing mechanical keeps its use read-only); both are settings to restate for your host.

Good to know

The catalogue scored this piece 3: the two files drop into any Claude Code repository's .claude/ unchanged, but the agent's 'Running inside the harness' section names the record's own harness (its plan gate, scratchpad convention and a global editable install), and the dispatch paragraphs that call it live inside harness-managed command files that are not copied.
METHOD.md says the critic's round reports are not committed; a recovered selection of them lives in the record's docs/audits/critic-recovered/, whose README says where the reports and the dispatch briefs lived. A workflow note for a reader outside the record (briefs, reports, verbatim saves) is still to be written before this can be a card.

The 3 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	LICENSE	8497cbc80c24	1070
sideband-critic.md	sideband/.claude/agents/sideband-critic.md	54b640ee2d3a	5675
skills/critic/SKILL.md	sideband/.claude/skills/critic/SKILL.md	182effe690b9	3456

lockstep integer coder

Lockstep coder: a pure-integer arithmetic coder that agrees bit for bit

lockstep · folder

25 files, 98804 bytes; MIT licence.

Contents: RangeEncoder; RangeDecoder; Cdf; quantize_weights; quantize_probs; deterministic_bytes; deterministic_ints; encode_bytes; decode_bytes; encode_stream; decode_stream; roundtrip_ok; make_model; load_vectors; run_vectors; fingerprint.

Tests of this copy: 33 passed

How to use this tool

An arithmetic coder written with whole numbers only, no floating point, so an encoder on one machine and a decoder on another produce and read exactly the same bytes. It comes with frequency tables, two byte models, a seeded byte generator and six conformance vectors that check a copy or a port agrees.

Copy the command under Get this folder and run it, then run python -m pytest in the new folder. It needs nothing beyond Python and pytest; conftest.py puts src on the path.
Keep the package folder under src whole: its __init__.py imports the archive layer (container.py and manifest.py) as well, so using the coder alone means trimming that file.
To compress bytes, call encode_bytes(data, make_model('adaptive-order1')), and keep the data's length: decode_bytes(payload, length, make_model('adaptive-order1')) needs it, with a fresh model on each side.
To code your own symbols, build a table with Cdf.from_freqs([3, 1, 4]), pass each symbol's Cdf.interval to RangeEncoder.encode, then call RangeEncoder.finish. To decode, give the bytes to RangeDecoder and, for each symbol, call RangeDecoder.decode_target with the table's total, look the result up with Cdf.find, and call RangeDecoder.update with that symbol's interval.
Call run_vectors() from lockstep.conformance to check this copy, or your own port, against the six pinned vectors; every result should come back ok.

A pure-integer arithmetic coder in stdlib Python with CDF quantizers, a seeded deterministic byte generator, an environment fingerprint and conformance vectors pinned by hash; for people who need an encoder and decoder to agree bit for bit across machines.

Get this folder

One command makes a new folder named lockstep-integer-coder inside the folder your terminal is in. It changes nothing else, joins no project and installs nothing. Paste it into bash, zsh or PowerShell with Python on the path; where python is not found, change the first word to python3. It downloads the 25 files listed below from the repository at tag v1.0.0, checks each against its full sha256, writes nothing unless every one matches and the folder is new, and adds the pytest.ini the tests run under. The tests need pytest: pip install pytest, then cd lockstep-integer-coder and python -m pytest.

Show the command

python -c "
import hashlib, pathlib, urllib.request
base = 'https://raw.githubusercontent.com/Mike-Byrne-AI/specdec-sidechannels/v1.0.0/'
out = pathlib.Path('lockstep-integer-coder')
files = [
    ('LICENSE', 'lockstep/LICENSE', '8497cbc80c24245fc8f9be355fca4a8d15b0235b7de4870b30d573189092aeef'),
    ('conformance/vectors_v0.json', 'lockstep/conformance/vectors_v0.json', 'c452e20b01344a9d4b1e0c0f42b17da97572de7ac52746a3c9ef6e91934b4861'),
    ('conftest.py', 'lockstep/conftest.py', '8931e98173a1dca5e11132f8db85317156bd78182ea173ce31845d3f36f81ef7'),
    ('specs/conformance-v0.md', 'lockstep/specs/conformance-v0.md', '8961c9816e7244403eb607fc7a0fee3634a3e455a8f75bade846079de3ac220c'),
    ('src/lockstep/__init__.py', 'lockstep/src/lockstep/__init__.py', 'fb19da5a4cf0dfcb9a2d510fcc080b359c56faadd59406c6cc94fa49209b13ed'),
    ('src/lockstep/cdf.py', 'lockstep/src/lockstep/cdf.py', 'dc3c9ba84bf328f4bae09f676b2c622f47d0d0f65c95a6494176586ab1cfba8f'),
    ('src/lockstep/codec.py', 'lockstep/src/lockstep/codec.py', '0f02b2215b5956d7cb8d8980e4deb94136dc9d08b88c99af362305a25778714f'),
    ('src/lockstep/conformance.py', 'lockstep/src/lockstep/conformance.py', '83639ee4664c51370724ea1c5f7ba495a97b644c8cf93d1df04c9c0c7a81c8c9'),
    ('src/lockstep/container.py', 'lockstep/src/lockstep/container.py', '81b0dc83f94c22d2f5af9d8931a28e7b8612869bcd00087b73e1282a08845703'),
    ('src/lockstep/data/__init__.py', 'lockstep/src/lockstep/data/__init__.py', '273c891b3cf71c48d52738a5455af07eae7de9c8708770e8a3c8c276d66c2930'),
    ('src/lockstep/data/vectors_v0.json', 'lockstep/src/lockstep/data/vectors_v0.json', 'c452e20b01344a9d4b1e0c0f42b17da97572de7ac52746a3c9ef6e91934b4861'),
    ('src/lockstep/drbg.py', 'lockstep/src/lockstep/drbg.py', 'c5367ba5b0306d03e30a51b8496c06eb9c674d9779d16b6580d6eb20116c6679'),
    ('src/lockstep/envprobe.py', 'lockstep/src/lockstep/envprobe.py', 'fa833d0f5be48b80a8da4272e9c11bb5cc89599e8aba3906127ced314f5c8f18'),
    ('src/lockstep/errors.py', 'lockstep/src/lockstep/errors.py', '755d94d1169565ec0aa5b7684147f2ff8b7c357db5b5cfe73f443bfdf717fc26'),
    ('src/lockstep/manifest.py', 'lockstep/src/lockstep/manifest.py', 'a00c7463d68d42e0809712a1cbbed52b33171260dea1a8ec84b7d0ae99581d3b'),
    ('src/lockstep/models.py', 'lockstep/src/lockstep/models.py', 'b40a55c06d8126ddf2f0bf697931c06d8ec4731cbf2215ee0dc1a03a47f42654'),
    ('src/lockstep/rac.py', 'lockstep/src/lockstep/rac.py', '9cc23cd7f817b6de88c3c17c52ed8f64f18aaba3cdaf9d4fd6e9132bab89a4aa'),
    ('tests_project/__init__.py', 'lockstep/tests_project/__init__.py', '8f5aebc872a6d88b5f9d29acd6df67a23d5e63cec146c42d642d18c55d7f4d78'),
    ('tests_project/test_api_validation.py', 'lockstep/tests_project/test_api_validation.py', 'ecbf0d2dec309ca06c21e2c742c8d7211c644adbea05050e11e4176754b12716'),
    ('tests_project/test_cdf.py', 'lockstep/tests_project/test_cdf.py', 'a7699d30d309d591e8223ee0a28c6ab450e36366677d4686f0673394a8ef5826'),
    ('tests_project/test_codec.py', 'lockstep/tests_project/test_codec.py', '6286642b6287b0128639db7e9ce5f011154519ef1d5faef22d8c0d51508a41e1'),
    ('tests_project/test_conformance.py', 'lockstep/tests_project/test_conformance.py', '6e0efa70663bed3e24dfb76a7169063c7ddbc9103c7e4b01ff213d516a3dda0d'),
    ('tests_project/test_packaged_data.py', 'lockstep/tests_project/test_packaged_data.py', '9719313d3436f8ead7719ff7a1e19a4cabc1f03cc1447967b6b20dddd7fe5cdd'),
    ('tests_project/test_rac.py', 'lockstep/tests_project/test_rac.py', '589358c8481a5eb244b9abcd50b3977eb6f3afc24864b69c26a4e7cd8a0631b1'),
    ('tools/lockstep_gen_vectors.py', 'lockstep/tools/lockstep_gen_vectors.py', '4840ba922d8fbc58113dbaed2a04fc968e2fd05d7751e9e5173c6dfb3ff786b9'),
]
if out.exists():
    raise SystemExit(str(out) + ' already exists: nothing written')
got = []
for dest, src, sha in files:
    data = urllib.request.urlopen(base + src).read()
    if hashlib.sha256(data).hexdigest() == sha:
        got.append((out / dest, data))
    else:
        raise SystemExit(src + ': its sha256 is not the one on the page: nothing written')
for path, data in got:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_bytes(data)
(out / 'pytest.ini').write_bytes(chr(10).join(['[pytest]', 'testpaths = tests_project', 'pythonpath = src', '']).encode())
print('checked and wrote', len(got), 'files to', out.resolve())
"

In the record

lockstep is the third folder of the release: deterministic arithmetic coding and model-pinned archives. The coder is the part that has to agree bit for bit between an encoder and a decoder on different machines, and the conformance vectors, pinned by hash in conformance.py and shipped twice (a data copy inside the package and one beside the spec), are how that agreement is checked.

Use

Copy src/lockstep/ whole: the package __init__ re-exports container.py and manifest.py (the archive layer), so they come along even if you only want the coder; trimming the __init__ is the edit to make if you do not.
Run python -m pytest from this folder; conftest.py puts src/ on the path as the record's own layout does.

Good to know

rac.py describes itself as a Witten-Neal-Cleary arithmetic coder while its classes are named RangeEncoder and RangeDecoder.
envprobe.py's docstring points at a command-line entry point that is not copied here.
The two vectors_v0.json copies must stay byte-identical; the manifest records both digests.

The 25 files, each linked to the same file at v1.0.0

Module path	Public file at v1.0.0	sha256	Bytes
LICENSE	lockstep/LICENSE	8497cbc80c24	1070
conformance/vectors_v0.json	lockstep/conformance/vectors_v0.json	c452e20b0134	3208
conftest.py	lockstep/conftest.py	8931e98173a1	89
specs/conformance-v0.md	lockstep/specs/conformance-v0.md	8961c9816e72	2937
src/lockstep/__init__.py	lockstep/src/lockstep/__init__.py	fb19da5a4cf0	1828
src/lockstep/cdf.py	lockstep/src/lockstep/cdf.py	dc3c9ba84bf3	6384
src/lockstep/codec.py	lockstep/src/lockstep/codec.py	0f02b2215b59	5455
src/lockstep/conformance.py	lockstep/src/lockstep/conformance.py	83639ee4664c	3772
src/lockstep/container.py	lockstep/src/lockstep/container.py	81b0dc83f94c	25814
src/lockstep/data/__init__.py	lockstep/src/lockstep/data/__init__.py	273c891b3cf7	47
src/lockstep/data/vectors_v0.json	lockstep/src/lockstep/data/vectors_v0.json	c452e20b0134	3208
src/lockstep/drbg.py	lockstep/src/lockstep/drbg.py	c5367ba5b030	1624
src/lockstep/envprobe.py	lockstep/src/lockstep/envprobe.py	fa833d0f5be4	2449
src/lockstep/errors.py	lockstep/src/lockstep/errors.py	755d94d11695	1287
src/lockstep/manifest.py	lockstep/src/lockstep/manifest.py	a00c7463d68d	5862
src/lockstep/models.py	lockstep/src/lockstep/models.py	b40a55c06d81	3693
src/lockstep/rac.py	lockstep/src/lockstep/rac.py	9cc23cd7f817	9565
tests_project/__init__.py	lockstep/tests_project/__init__.py	8f5aebc872a6	164
tests_project/test_api_validation.py	lockstep/tests_project/test_api_validation.py	ecbf0d2dec30	4835
tests_project/test_cdf.py	lockstep/tests_project/test_cdf.py	a7699d30d309	2837
tests_project/test_codec.py	lockstep/tests_project/test_codec.py	6286642b6287	1983
tests_project/test_conformance.py	lockstep/tests_project/test_conformance.py	6e0efa70663b	2201
tests_project/test_packaged_data.py	lockstep/tests_project/test_packaged_data.py	9719313d3436	1156
tests_project/test_rac.py	lockstep/tests_project/test_rac.py	589358c8481a	3550
tools/lockstep_gen_vectors.py	lockstep/tools/lockstep_gen_vectors.py	4840ba922d8f	3786