ruuda
commented
6 years ago Sorry about the confusing benchmarks, benches/testsamples.rs is a remnant that I should remove. There are tools/benchmark.sh to run the benchmark and tools/compare_benches.r to compare two runs.
Usage is as follows:
# Put a few flac files in testsamples/extra.
# I use five files, all 44100 Hz, 16 bit, stereo (CD quality), together 193 MiB.
git checkout master
tools/benchmark.sh before
git checkout alternative
tools/benchmark.sh after
tools/compare_benches.r before_*_all.dat after_*_all.datThat will print a table like this:
| metric | value after (mean and stddev) | fraction of before |
|---|---|---|
| p10 (0.1 quantile time per sample) | 18.1 ± 0.2 ns | 1.000 ± 0.016 |
| p50 (median time per sample) | 18.9 ± 0.3 ns | 1.000 ± 0.019 |
| p90 (0.9 quantile time per sample) | 20.0 ± 0.4 ns | 0.996 ± 0.040 |
| μ (mean time per sample) | 19.2 ± 0.3 ns | 0.999 ± 0.020 |
| τ (input bytes throughput) | 66.7 ± 3.4 MiB/s | 1.001 ± 0.074 |
Depending on the kind of input file, this change is indeed not going to make a difference, because the buffer is recycled and allocated with the right size immediately. To test the worst case impact, you can replace this line to always pass in a clean Vec::new(), which will be too small, triggering ensure_buffer_len to allocate a bigger one on every block.
However, the performance difference between Claxon master and libflac 1.3.1 is 1.44, not 1.13 as advertised in README.md; it is possible that on my machine the execution is bottlenecked elsewhere, and I would not notice performance degradation caused by this PR. So please run this through your benchmarking setup before merging.
This is interesting ... when I run Claxon again now on the same system, I get about 1.4 times libflac too, even on Claxon 0.3.0. One thing that always stood out was that although Claxon was close to libflac in running time, it was executing many more instructions, and indeed on my Raspberry Pi it was not within a factor 2 of libflac if I recall correctly. So if Claxon was almost as fast as libflac, it was through the CPU being able to deal with those extra instructions, not because it did as little work as libflac. It might be that we are seeing the effects of Meltdown/Spectre/Foreshadow mitigations here.
Shnatsel
This should be the fastest way to safely initialize an intermediate buffer.
vec![value; lenght]desugars intoVec::from_elem()which has a fast path for value 0 that requests zeroed memory from the OS, and the OS zeroes memory on demand. Addresses #13.I have converted the benchmarking suite to Criterion to get more accurate benchmarks, specifically so that I would be able to reliably detect <2% changes in performance which is impossible with regular cargo-bench. It also has an added bonus of working on stable Rust. I can open a PR for that if you'd like to have that in master.
According to the
testsamples.rsbenchmark, as well as measuring execution time of the bench_decode binary with hyperfine, this change makes no difference in terms of performance. I have tested this both onrustc 1.28.0 (9634041f0 2018-07-30)andrustc 1.30.0-nightly (d767ee116 2018-08-15)with identical results.However, the performance difference between Claxon master and libflac 1.3.1 is 1.44, not 1.13 as advertised in README.md; it is possible that on my machine the execution is bottlenecked elsewhere, and I would not notice performance degradation caused by this PR. So please run this through your benchmarking setup before merging.