Does this work with "rayon" and "tokio"?

[firasuke]

Does xxhash-rust work with rayon and/or tokio, particularly when reading an entire file into a buffer, then hashing it (is it possible to perform the xxh3 hash in parallel?) (or maybe hash multiple files read into buffers at the same time?)?

Thanks in advance!

[DoumanAsh]

no, there is no parallelization. If you read entire file into buffer you do not need to rayon or tokio.

Afaik in theory hash can be performed in parallel especially if whole buffer is available ahead, but this is in fact what is done with HW instructions partly as it processes much bigger chunks of data, especially if you compile with AVX instructions available.

But otherwise no, there is no special implementation for that and it would be likely ineffective unless your input is really big

[DoumanAsh]

As a side note: reading whole file into memory in no shape or form is efficient solution. You're better off to read it by chunks and feed into streaming hasher unless file is really small

[firasuke]

I see, thanks for the informative reply!