Continuing the project

[shainer]

Hello,

this is not the best platform for this discussion but I had no idea how to contact you otherwise. I am a developer with an interest in cryptography and a (relatively recent) interest in Rust.

I'd like to contribute to this project; I need to sort out a couple of things before I can be 100% sure of this, but it's likely. In the meantime I forked the repo and I am going to play around with the code; mostly writing tests or more examples to get familiar with the codebase.

What do you think?

[newpavlov]

Take a look at RustCrypto organization, it aims to succeed rust-crypto. Currently it developed a bit slowly (not all algorithms from rust-crypto were ported yet), but steadily. And of course contributions are always welcomed!

[kkartaltepe]

There is also work going on in the ring project which seems to cover a good portion of the functionality covered in rust-crypto. Im not sure if anyone has compared the various crypto libraries but maybe that would be worth finding or creating.

[newpavlov]

@kkartaltepe There is crypto-bench from the ring author to compare performance of different crypto libraries.

[kkartaltepe]

I actually just found the crypto-bench project, but they really should post some results from an average machine.

In my own testing today I found ring lacking in my particular use-case of simple sha1 digests. In my application ring is about twice as slow. This looks to be consistent with the crypto-bench maintainers own results from a sample I found https://gist.github.com/briansmith/9b28c6033f3f16626e8bed097da3fbd1

I know when i looked rust-crypto had a lot of love put in to being fast and im not sure any of the other implementations are really competitive yet. Maybe crypto-bench can be further expanded in the future to be a good survey of the ecosystem.

[briansmith]

@kkartaltepe ring's SHA-1 is optimized for size, because there's only one valid use case for SHA-1 that I know of, which is processing OCSP responses that currently mandate SHA-1. Pick any other digest algorithm and the performance story is completely different.

[kkartaltepe]

Well if you want to broaden your horizons, sha1 is the hashing algorithm used for integrity checking for BitTorrent pieces. And while both ring and rust-crypto are far faster than spinning disks, the difference in hashing speed makes an appreciable difference when you have constrained IO buffers.

Unfortunately, I don't see this standard changing any time soon.

[briansmith]

@kkartaltepe First, I think we agree the ring implementation of SHA-1 isn't fast. But I suspect it is faster enough for typical BitTorrent clients. Consider that the compression that is probably used for the file is almost definitely more expensive than even a bad SHA-1.

[kkartaltepe]

I agree ring is fast enough, and undoubtedly there are improvements I could make to my own code outside the realm of which SHA-1 implementation I chose. Of course being optimized for size I wouldn't expect ring to be as fast as something optimized for speed.

But my use case is trying to compete with C/C++ torrent libraries where speed is a factor in user perception. On fast disks or in cases where data is cached in memory the hashing algorithm can become the bottleneck and a difference in speed becomes quite apparent to the user. These cases often happen when users choose to or are forced to re-verify the integrity of their files and long hash times are already a major annoyance even in fast C/C++ implementations. Even with rust-crypto I am only able to achieve ~90% of the hashing performance when I compare my code to C++ implementations.

I think ring is quite a well developed library and I'm excited to see it grow, especially now that rust-crypto is seemingly abandoned. I hope to really try it out if I find myself doing real crypto work in the future, but for my current fast digest needs I cant justify switching.

[newpavlov]

@kkartaltepe Try to use sha-1 crate, on my machine it processes 490 MB/s and a bit faster with enabled asm feature. As for comparison with C++ I think reason behind such difference is hardware acceleration, which can be used on the stable Rust only through gcc. If you'll have time and expertise you could contribute to asm-hashes/sha1 with hardware accelerated code. You can read comments in the sha-1 source code (inherited from rust-crypto) for the reference.

[shainer]

Thanks everybody for the pointers on the projects. I am away until the end of the week so I'll get back to this in a few days. I'd love to contribute to ongoing efforts rather than duplicate work if possible.

[shainer]

@newpavlov why did you split the rust-crypto functionalities in so many repositories? I understand separating fuzz tests, but I see a repo for block ciphers, stream ciphers, and hashes, which I'd consider basic features to include in one single crypto library. Doesn't this make it harder to add things like CI or linting? Is it a style preference in the Rust world?

[newpavlov]

@shainer Well it's a matter of approach, as one of the goals for RustCrypto is to create an ecosystem of crates, not one library. So you could just add say only HMAC-SHA2 and AES-GCM to your project and not the whole library with a lot of features which you don't need. Also I think division of crates into thematic repositories will help in future with managing development process. Thanks to cargo this approach works without much problems. (there is several minor drawbacks, but they are manageable) As for CI currently it tested using cargo test --all which works thanks to the fact that all crates in repository are part of one workspace. But future CI setup will be more complex as we'll need to test different features and different architectures.

I wouldn't call such approach a widespread one, because not many projects have prerequisites for it. It will be honest to mention that some projects (namely winapi crates) move back to monolithic approach.

[spacekookie]

@newpavlov In general, having looked at the RustCrypto organisation and what's in it, all the implementations seem very low level. You don't have a function that encrypts some data with a block cipher, you have functions to do it block by block where any block chaining is left up to you.

I think, especially for people not super familiar with cryptography, just wanting a crate that "encrypts data" is kind of an important use case. And there, apart from ring, I haven't really seen anything in the rust ecosystem :/ Or did I miss something?

[newpavlov]

@spacekookie The main goal of RustCrypto is to provide modular foundation for Rust crypto ecosystem. We definitely have plans for higher level algorithms, but first we need to work out basics. (I plan to work on ciphers after finalization of digest v0.8, but unfortunately I couldn't find enough time lately) In addition to ring and various bindings to OpenSSL and sodiumoxide I can recommend to take a look at miscreant project, which uses several crates from RustCrypto.