bheisler/criterion.rs
### [`v0.5.1`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#051---2023-05-26)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.5.0...0.5.1)
##### Fixed
- Quick mode (--quick) no longer crashes with measured times over 5 seconds when --noplot is not active
### [`v0.5.0`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#050---2023-05-23)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.4.0...0.5.0)
##### Changed
- Replaced lazy_static dependency with once_cell
- Improved documentation of the `html_reports` feature
- Replaced atty dependency with is-terminal
- MSRV bumped to 1.64
- Upgraded clap dependency to v4
- Upgraded tempfile dependency to v3.5.0
##### Fixed
- Quick mode (`--quick`) no longer outputs 1ms for measured times over 5 seconds
- Documentation updates
### [`v0.4.0`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#040---2022-09-10)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.6...0.4.0)
##### Removed
- The `Criterion::can_plot` function has been removed.
- The `Criterion::bench_function_over_inputs` function has been removed.
- The `Criterion::bench_functions` function has been removed.
- The `Criterion::bench` function has been removed.
##### Changed
- HTML report hidden behind non-default feature flag: 'html_reports'
- Standalone support (ie without cargo-criterion) feature flag: 'cargo_bench_support'
- MSRV bumped to 1.57
- `rayon` and `plotters` are optional (and default) dependencies.
- Status messages ('warming up', 'analyzing', etc) are printed to stderr, benchmark results are printed to stdout.
- Accept subsecond durations for `--warm-up-time`, `--measurement-time` and `--profile-time`.
- Replaced serde_cbor with ciborium because the former is no longer maintained.
- Upgrade clap to v3 and regex to v1.5.
##### Added
- A `--discard-baseline` flag for discarding rather than saving benchmark results.
- Formal support for benchmarking code compiled to web-assembly.
- A `--quiet` flag for printing just a single line per benchmark.
- A `Throughput::BytesDecimal` option for measuring throughput in bytes but printing them using
decimal units like kilobytes instead of binary units like kibibytes.
##### Fixed
- When using `bench_with_input`, the input parameter will now be passed through `black_box` before
passing it to the benchmark.
### [`v0.3.6`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#036---2022-07-06)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.5...0.3.6)
##### Changed
- MSRV bumped to 1.49
- Symbol for microseconds changed from ASCII 'us' to unicode 'µs'
- Documentation fixes
- Clippy fixes
### [`v0.3.5`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#035---2021-07-26)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.4...0.3.5)
##### Fixed
- Corrected `Criterion.toml` in the book.
- Corrected configuration typo in the book.
##### Changed
- Bump plotters dependency to always include a bug-fix.
- MSRV bumped to 1.46.
### [`v0.3.4`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#034---2021-01-24)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.3...0.3.4)
##### Added
- Added support for benchmarking async functions
- Added `with_output_color` for enabling or disabling CLI output coloring programmatically.
##### Fixed
- Criterion.rs will now give a clear error message in case of benchmarks that take zero time.
- Added some extra code to ensure that every sample has at least one iteration.
- Added a notice to the `--help` output regarding "unrecognized option" errors.
- Increased opacity on violin charts.
- Fixed violin chart X axis not starting at zero in the plotters backend.
- Criterion.rs will now automatically detect the right output directory.
##### Deprecated
- `Criterion::can_plot` is no longer useful and is deprecated pending deletion in 0.4.0.
- `Benchmark` and `ParameterizedBenchmark` were already hidden from documentation, but are now
formally deprecated pending deletion in 0.4.0. Callers should use `BenchmarkGroup` instead.
- `Criterion::bench_function_over_inputs`, `Criterion::bench_functions`, and `Criterion::bench` were
already hidden from documentation, but are now formally deprecated pending deletion in 0.4.0.
Callers should use `BenchmarkGroup` instead.
- Three new optional features have been added; "html_reports", "csv_output" and
"cargo_bench_support". These features currently do nothing except disable a warning message at
runtime, but in version 0.4.0 they will be used to enable HTML report generation, CSV file
generation, and the ability to run in cargo-bench (as opposed to [cargo-criterion]).
"cargo_bench_support" is enabled by default, but "html_reports" and "csv_output"
are not. If you use Criterion.rs' HTML reports, it is recommended to switch to [cargo-criterion].
If you use CSV output, it is recommended to switch to [cargo-criterion] and use the
`--message-format=json` option for machine-readable output instead. A warning message will be
printed at the start of benchmark runs which do not have "html_reports" or "cargo_bench_support"
enabled, but because CSV output is not widely used it has no warning.
[cargo-criterion]: https://togithub.com/bheisler/cargo-criterion
### [`v0.3.3`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#033---2020-06-29)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.2...0.3.3)
##### Added
- Added `CRITERION_HOME` environment variable to set the directory for Criterion to store
its results and charts in.
- Added support for \[cargo-criterion]. The long-term goal here is to remove code from Criterion-rs
itself to improve compile times, as well as to add features to `cargo-criterion` that are
difficult to implement in Criterion-rs.
- Add sampling mode option for benchmarks. This allows the user to change how Criterion.rs chooses
the iteration counts in each sample. By default, nothing will change for most benchmarks, but
very slow benchmarks will now run fewer iterations to fit in the desired number of samples.
This affects the statistics and plots generated.
##### Changed
- The serialization format for some of the files has changed. This may cause your first benchmark
run after updating to produce errors, but they're harmless and will go away after running the
benchmarks once.
##### Fixed
- Fixed a bug where the current measurement was not shown on the relative regression plot.
- Fixed rare panic in the plotters backend.
- Panic with a clear error message (rather than panicking messily later on) when the user sets the
group or function name to the empty string.
- Escape single quotes in benchmark names when generating Gnuplot scripts.
### [`v0.3.2`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#032---2020-04-26)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.1...0.3.2)
##### Added
- Added `?Sized` bound to benchmark parameter types, which allows dynamically sized types like
`&str` and `&[T]` to be used as benchmark parameters.
- Added the `--output-format ` command-line option. If `--output-format bencher` is passed,
Criterion.rs will print its measurements in a format similar to that used by the `bencher` crate
or unstable `libtest` benchmarks, and using similar statistical measurements as well. Though this
provides less information than the default format, it may be useful for supporting tools which
parse this output format.
- Added `--nocapture` argument. This argument does nothing, but prevents Criterion.rs from exiting
when running tests or benchmarks and allowing stdout output from other tests.
##### Fixed
- Fixed panic when environment variables contains non-UTF8 characters.
- Fixed panic when `CRITERION_DEBUG` or `CRITERION_TARGET_DIR` environment variables contain
non-UTF8 characters.
### [`v0.3.1`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#031---2020-01-25)
[Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.0...0.3.1)
##### Added
- Added new plotting backend using the `plotters` crate. Implementation generously provided by Hao
Hou, author of the `plotters` crate.
- Added `--plotting-backend` command-line option to select the plotting backend. The existing
gnuplot backend will be used by default when available, and the plotters backend will be used when
gnuplot is not available or when requested.
- Added `Criterion::plotting_backend()` function to configure the plotting backend in code.
- Added `--load-baseline` command-line option to load a baseline for comparison
rather than measuring the current code
- Benchmark filters can now be regular expressions.
##### Fixed
- Fixed `fibonacci` functions.
- Fixed `#[criterion]` benchmarks ignoring the command-line options.
- Fixed incorrect scaling of the violin plots.
- Don't print the recommended sample count if it's the same as the configured
sample count.
- Fix potential panic when `nresamples` is set too low. Also added a warning
against setting `nresamples` too low.
- Fixed issue where a slow outer closure would cause Criterion.rs to calculate
the wrong estimated time and number of iterations in the warm-up phase.
Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
[ ] If you want to rebase/retry this PR, check this box
This PR has been generated by Mend Renovate. View repository job log here.
This PR contains the following updates:
0.3
->0.5
Release Notes
bheisler/criterion.rs
### [`v0.5.1`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#051---2023-05-26) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.5.0...0.5.1) ##### Fixed - Quick mode (--quick) no longer crashes with measured times over 5 seconds when --noplot is not active ### [`v0.5.0`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#050---2023-05-23) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.4.0...0.5.0) ##### Changed - Replaced lazy_static dependency with once_cell - Improved documentation of the `html_reports` feature - Replaced atty dependency with is-terminal - MSRV bumped to 1.64 - Upgraded clap dependency to v4 - Upgraded tempfile dependency to v3.5.0 ##### Fixed - Quick mode (`--quick`) no longer outputs 1ms for measured times over 5 seconds - Documentation updates ### [`v0.4.0`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#040---2022-09-10) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.6...0.4.0) ##### Removed - The `Criterion::can_plot` function has been removed. - The `Criterion::bench_function_over_inputs` function has been removed. - The `Criterion::bench_functions` function has been removed. - The `Criterion::bench` function has been removed. ##### Changed - HTML report hidden behind non-default feature flag: 'html_reports' - Standalone support (ie without cargo-criterion) feature flag: 'cargo_bench_support' - MSRV bumped to 1.57 - `rayon` and `plotters` are optional (and default) dependencies. - Status messages ('warming up', 'analyzing', etc) are printed to stderr, benchmark results are printed to stdout. - Accept subsecond durations for `--warm-up-time`, `--measurement-time` and `--profile-time`. - Replaced serde_cbor with ciborium because the former is no longer maintained. - Upgrade clap to v3 and regex to v1.5. ##### Added - A `--discard-baseline` flag for discarding rather than saving benchmark results. - Formal support for benchmarking code compiled to web-assembly. - A `--quiet` flag for printing just a single line per benchmark. - A `Throughput::BytesDecimal` option for measuring throughput in bytes but printing them using decimal units like kilobytes instead of binary units like kibibytes. ##### Fixed - When using `bench_with_input`, the input parameter will now be passed through `black_box` before passing it to the benchmark. ### [`v0.3.6`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#036---2022-07-06) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.5...0.3.6) ##### Changed - MSRV bumped to 1.49 - Symbol for microseconds changed from ASCII 'us' to unicode 'µs' - Documentation fixes - Clippy fixes ### [`v0.3.5`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#035---2021-07-26) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.4...0.3.5) ##### Fixed - Corrected `Criterion.toml` in the book. - Corrected configuration typo in the book. ##### Changed - Bump plotters dependency to always include a bug-fix. - MSRV bumped to 1.46. ### [`v0.3.4`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#034---2021-01-24) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.3...0.3.4) ##### Added - Added support for benchmarking async functions - Added `with_output_color` for enabling or disabling CLI output coloring programmatically. ##### Fixed - Criterion.rs will now give a clear error message in case of benchmarks that take zero time. - Added some extra code to ensure that every sample has at least one iteration. - Added a notice to the `--help` output regarding "unrecognized option" errors. - Increased opacity on violin charts. - Fixed violin chart X axis not starting at zero in the plotters backend. - Criterion.rs will now automatically detect the right output directory. ##### Deprecated - `Criterion::can_plot` is no longer useful and is deprecated pending deletion in 0.4.0. - `Benchmark` and `ParameterizedBenchmark` were already hidden from documentation, but are now formally deprecated pending deletion in 0.4.0. Callers should use `BenchmarkGroup` instead. - `Criterion::bench_function_over_inputs`, `Criterion::bench_functions`, and `Criterion::bench` were already hidden from documentation, but are now formally deprecated pending deletion in 0.4.0. Callers should use `BenchmarkGroup` instead. - Three new optional features have been added; "html_reports", "csv_output" and "cargo_bench_support". These features currently do nothing except disable a warning message at runtime, but in version 0.4.0 they will be used to enable HTML report generation, CSV file generation, and the ability to run in cargo-bench (as opposed to [cargo-criterion]). "cargo_bench_support" is enabled by default, but "html_reports" and "csv_output" are not. If you use Criterion.rs' HTML reports, it is recommended to switch to [cargo-criterion]. If you use CSV output, it is recommended to switch to [cargo-criterion] and use the `--message-format=json` option for machine-readable output instead. A warning message will be printed at the start of benchmark runs which do not have "html_reports" or "cargo_bench_support" enabled, but because CSV output is not widely used it has no warning. [cargo-criterion]: https://togithub.com/bheisler/cargo-criterion ### [`v0.3.3`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#033---2020-06-29) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.2...0.3.3) ##### Added - Added `CRITERION_HOME` environment variable to set the directory for Criterion to store its results and charts in. - Added support for \[cargo-criterion]. The long-term goal here is to remove code from Criterion-rs itself to improve compile times, as well as to add features to `cargo-criterion` that are difficult to implement in Criterion-rs. - Add sampling mode option for benchmarks. This allows the user to change how Criterion.rs chooses the iteration counts in each sample. By default, nothing will change for most benchmarks, but very slow benchmarks will now run fewer iterations to fit in the desired number of samples. This affects the statistics and plots generated. ##### Changed - The serialization format for some of the files has changed. This may cause your first benchmark run after updating to produce errors, but they're harmless and will go away after running the benchmarks once. ##### Fixed - Fixed a bug where the current measurement was not shown on the relative regression plot. - Fixed rare panic in the plotters backend. - Panic with a clear error message (rather than panicking messily later on) when the user sets the group or function name to the empty string. - Escape single quotes in benchmark names when generating Gnuplot scripts. ### [`v0.3.2`](https://togithub.com/bheisler/criterion.rs/blob/HEAD/CHANGELOG.md#032---2020-04-26) [Compare Source](https://togithub.com/bheisler/criterion.rs/compare/0.3.1...0.3.2) ##### Added - Added `?Sized` bound to benchmark parameter types, which allows dynamically sized types like `&str` and `&[T]` to be used as benchmark parameters. - Added the `--output-formatConfiguration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR has been generated by Mend Renovate. View repository job log here.