search

buildbarn / bb-storage

Storage daemon, capable of storing data for the Remote Execution protocol
Apache License 2.0
136 stars 91 forks source link

blake3 support #207

Open nate-thirdwave opened 2 months ago

nate-thirdwave commented 2 months ago

Using startup --digest_function=blake3 for bazel invocations yields execution results that are unparseable by bb-browser: image

blake3 is much faster than sha256.

EdSchouten commented 2 months ago

blake3 is much faster than sha256.

Only in certain cases. I did some benchmarking on an Apple M1 CPU some time ago, and I wasn’t able to achieve a throughput with BLAKE3 that was anywhere near SHA-256.

BLAKE3 is only faster on CPU architectures that provide wide SIMD support (e.g., AVX2), but don’t offer native SHA-256 instructions. And there it’s also only faster for large inputs. In the case of REv2 we see that clients must do lots of hashing of small inputs (<1KB), meaning that BLAKE3 is a bad fit.

That said, if people really want to see BLAKE3 support being added to Buildbarn, feel free to submit a PR. Changes need to be made to bb-storage/pkg/digest.