BLAKE3-team / BLAKE3

the official Rust and C implementations of the BLAKE3 cryptographic hash function
Apache License 2.0
4.71k stars 315 forks source link

Please provide slimmed down version of b3sum #363

Open Hermholtz opened 7 months ago

Hermholtz commented 7 months ago

Hello. Thanks for implementing b3sum, much needed for many purposes. I like it, it's working and all, written in trendy Rust etc., but the binary size is 1,104,376 bytes (Ubuntu 23.10), which is among the largest binaries in my /usr/bin. The bash is 1.4 MiB, and it's a complete shell with a programming language.

For embedded machines, such as Raspberry Pi, or slimmed-down containers or many other applications we'd need a single-threaded C version of b3sum officially distributed and supported.

For your reference: b2sum is only 51744 bytes, sha*sum and md*sum are also 40-50 KiB in size.

Please provide that.

Thank you.

oconnor663 commented 7 months ago

I can get it down to ~800 KB using these steps: https://github.com/johnthagen/min-sized-rust

It's possible we could save some more space by making features like Rayon and Mmap optional (though those are crucial for performance). It might also be that Clap takes up more binary space that we would prefer, though it's a bit more work to test that.

The best space-saving strategy is probably to come up with a patch for busybox to provide a b3sum command there?

Hermholtz commented 7 months ago

The "example.c" which can calculate the sum compiles to an executable with length 84104. I think with additional ~10K it could parse command line and that's what's needed. Also, SSE versions could be removed, only AVX could be preserved. They wouldn't apply to ARM and many embedded targets anyway. Thanks.

ultrabear commented 7 months ago

If clap size is a concern, argh might be a good replacement, though it lacks functionality tried upx? that normally shrinks stuff, though it has broken on some of my rust code before

I dont think making a C implementation alongside the existing rust implementation is worth anyones time, you can totally write small rust code if you put the time in to use libc directly and handwrite more things (even when that can lead to more bugs), but that doesn't require C

nirs commented 4 months ago

Since we already have a C implementation which seems to be fast as single threaded Rust one, I think the best way to make it compact and available everywhere is to package the C version as a library (like it was done in Fedora), and integrate it in coreutils: https://github.com/coreutils/coreutils/blob/master/src/digest.c

This version will slower and provides less features, but it will be on every machine and likely to be integrated in busybox.

When coreutils is rewritten in rust it can use the rust crate :-)

lulcat commented 1 month ago

here there is a b2/b3 combined binary (C) of 100k. if you can't write the simple example.c to your needs, there is also https://github.com/michaelforney/b3sum (Note, I haven't tested it)