xxHash Checksum Verification

[BrandonMFong]

Hello,

I am comparing your xxHash checksum verification with the xxHash checksum tool created by the xxHash developers. The following is my output:

[OTASHI] sources\>_ ./repo/xxHash/xxhsum                                
Wrong parameters

xxhsum 0.8.1 by Yann Collet 
Print or verify checksums using fast non-cryptographic algorithm xxHash 

Usage: xxhsum [options] [files] 

When no filename provided or when '-' is provided, uses stdin as input. 
Options: 
  -H#         algorithm selection: 0,1,2 or 32,64,128 (default: 1) 
  -c, --check read xxHash checksum from [files] and check them 
  -h, --help  display a long help page about advanced options 
[OTASHI] sources\>_ ./repo/xxHash/xxhsum -H32 test.txt                  
d68e05bd  test.txt
[OTASHI] sources\>_ ./repo/xxHash/xxhsum -H64 test.txt
c8255a431a0f4393  test.txt
[OTASHI] sources\>_ ./repo/xxHash/xxhsum -H128 test.txt
2fef1828b95c3ffc57ddf3d57d095a29  test.txt
[OTASHI] sources\>_ ./mhl hash -vv -f test.txt -h xxHash d68e05bd        
Arguments error: Incorrect length of xxHash HASH value

Usage: 
mhl hash [-v | -vv]  -f FILE -h [md5|sha1] HASH
mhl hash [-v | -vv]  [-m] [-#] [-t] [md5|sha1] FILEPATTERNS...

[OTASHI] sources\>_ ./mhl hash -vv -f test.txt -h xxHash c8255a431a0f4393
Arguments error: Incorrect length of xxHash HASH value

Usage: 
mhl hash [-v | -vv]  -f FILE -h [md5|sha1] HASH
mhl hash [-v | -vv]  [-m] [-#] [-t] [md5|sha1] FILEPATTERNS...

[OTASHI] sources\>_ ./mhl hash -vv -f test.txt -h xxHash 2fef1828b95c3ffc57ddf3d57d095a29
Arguments error: Incorrect length of xxHash HASH value

Usage: 
mhl hash [-v | -vv]  -f FILE -h [md5|sha1] HASH
mhl hash [-v | -vv]  [-m] [-#] [-t] [md5|sha1] FILEPATTERNS...

[OTASHI] sources\>_ ./mhl hash -vv -f test.txt -h xxHash64 c8255a431a0f4393              
----------------------
Started checking XX64 hash for the file 'test.txt'
   with the size of 0 MB (12 bytes)
----------------------
Check XX64 hash for file 'test.txt' failed.
Passed hash sum does not match to calculated hash: 93430f1a435a25c8
----------------------
Finished checking XX64 hash for the file 'test.txt'
   with the size of 0 MB (12 bytes)
----------------------
Summary: FAILED

The contents of the test.txt file is:

[OTASHI] sources\>_ cat test.txt 
Hello World

I do realize that your help does not expose the xxHash|xxHash64 options:

[OTASHI] sources\>_ ./mhl hash                                             
Arguments error: Incorrect number of arguments.

Usage: 
mhl hash [-v | -vv]  -f FILE -h [md5|sha1] HASH
mhl hash [-v | -vv]  [-m] [-#] [-t] [md5|sha1] FILEPATTERNS...

[OTASHI] sources\>_ ./mhl hash -vv -f test.txt -h xxHash6 # Purposely passing typo            
Arguments error: Hash value doesn't has the length of MD5, SHA1, xxHash or xxHash64

Usage: 
mhl hash [-v | -vv]  -f FILE -h [md5|sha1] HASH
mhl hash [-v | -vv]  [-m] [-#] [-t] [md5|sha1] FILEPATTERNS...

[OTASHI] sources\>_ 

I don't know if this was as designed or a bug.

Please let me know if you have any questions

Best, Brando

[Mike-Woolley]

Yes I noticed this too.

I looked at the code and It seems mhl writes the hash string out in little endian, whereas xxh64sum uses big endian.

So you can either tell mhl to use big endian by specifying the hash as (the undocumented) xxhash64be or tell xxh64sum to use little endian via its --little-endian flag.

I hope that saves anyone else wasting time figuring out why they're different 😊

Mike

[kos-pomfort]

Hi.

There has been a historical confusion with little- and big-endian xxhash64 hash modes in mhl, which is also the reason why both modes are undocumented. As @Mike-Woolley said, use xxhash64be to get the (intended) big-endian hash.