CLI reference¶

pybench runs benchmarks by default (no subcommand needed), with update and show subcommands for managing baselines.

Default — discover & run benchmarks¶

pybench [PATH] [OPTIONS]

Option	Type	Default	Description
`PATH`	`str`	`./benchmarks`	Directory or file to scan for `bench_*` functions
`--baseline`	`str`	`.pybench/baselines.jsonl`	Path to the JSONL baseline store
`--bench`	`str`	—	Run only this benchmark (repeatable)
`--alpha`	`float`	—	Override `alpha` significance threshold
`--min-effect`	`float`	—	Override minimum relative drop to flag
`-v, --verbose`	flag	off	Expand per-metric per-step breakdown on failures
`--yes`	flag	off	Skip the dirty-tree confirmation prompt (for CI)

Discovered benchmarks with an existing baseline are re-run on the same stored seeds and compared statistically. Benchmarks without a baseline record are run, saved, and marked NEW. Exit code is 1 if any benchmark FAILs (like pytest).

Subcommands¶

pybench¶

Statistical benchmark regression detection (pybench).

Usage

pybench [OPTIONS] COMMAND [ARGS]...

show¶

Print current baseline stats, or their history with –history.

Usage

pybench show [OPTIONS]

Options

--baseline <baseline_path>¶: Path to JSONL store.

--bench <benches>¶: Show only this benchmark.

--history¶: Show baseline history from git.

update¶

Re-run benchmarks and overwrite their baseline records.

Usage

pybench update [OPTIONS] [PATH]

Options

--baseline <baseline_path>¶: Path to JSONL store.

--bench <benches>¶: Update only this benchmark.

--n-seeds <n_seeds>¶: Resample this many fresh seeds.

--yes¶: Skip confirmation prompts (for CI).

Arguments

PATH¶: Optional argument