charliermarsh
commented
8 months ago So I hooked up our download-and-unzip thing to also compute the SeaHash as it goes. It turns out that it doesn't affect the benchmarks at all:
❯ python -m scripts.bench \
--puffin-path ./target/release/puffin --puffin-path ./target/release/main \
scripts/requirements/compiled/trio.txt --min-runs 50 --benchmark install-cold
Benchmark 1: ./target/release/puffin (install-cold)
Time (mean ± σ): 627.8 ms ± 40.3 ms [User: 303.0 ms, System: 675.3 ms]
Range (min … max): 557.7 ms … 799.8 ms 50 runs
Benchmark 2: ./target/release/main (install-cold)
Time (mean ± σ): 627.5 ms ± 31.9 ms [User: 300.3 ms, System: 677.3 ms]
Range (min … max): 579.2 ms … 736.5 ms 50 runs
Summary
'./target/release/main (install-cold)' ran
1.00 ± 0.08 times faster than './target/release/puffin (install-cold)'
puffin on charlie/sha:main [$!⇡] is 📦 v0.0.3 via 🐍 v3.12.0 (.venv) via 🦀 v1.75.0 took 2m7s
❯ python -m scripts.bench \
--puffin-path ./target/release/puffin --puffin-path ./target/release/main \
scripts/requirements/compiled/black.txt --min-runs 50 --benchmark install-cold
Benchmark 1: ./target/release/puffin (install-cold)
Time (mean ± σ): 218.3 ms ± 37.9 ms [User: 61.1 ms, System: 78.3 ms]
Range (min … max): 193.4 ms … 472.6 ms 50 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Benchmark 2: ./target/release/main (install-cold)
Time (mean ± σ): 222.8 ms ± 28.0 ms [User: 63.2 ms, System: 82.4 ms]
Range (min … max): 198.6 ms … 332.3 ms 50 runs
Warning: Statistical outliers were detected. Consider re-running this benchmark on a quiet PC without any interferences from other programs. It might help to use the '--warmup' or '--prepare' options.
Summary
'./target/release/puffin (install-cold)' ran
1.02 ± 0.22 times faster than './target/release/main (install-cold)'Which is good (it's dominated by IO).
I'll try SHA256 too.
Make the archive bucket content-addressed for artifacts that have them.