satazor / js-spark-md5

Lightning fast normal and incremental md5 for javascript
Do What The F*ck You Want To Public License
2.46k stars 470 forks source link

Fix getState for rolling hashes #56

Closed overlookmotel closed 4 years ago

overlookmotel commented 4 years ago

This PR fixes a bug when calculating rolling hashes (i.e. get hash of data so far, and then continue to append more data).

One use case for rolling hashes is uploading large files to Google Drive via API. The file is transferred in chunks, and the server response after each chunk contains the MD5 hash of all data received up to that point. It's useful to be able to verify that hash on client and then continue appending data to the hash when sending following chunks.

More background on the use case: https://github.com/nodejs/node/issues/29903

Test case:

// One shot
const hash1 = SparkMD5.hash('Hi there');
console.log('hash1:', hash1);

// Incremental
const spark2 = new SparkMD5();
spark2.append('Hi');
spark2.append(' there');
const hash2 = spark2.end(); 
console.log('hash2:', hash2);

// Rolling
const spark3 = new SparkMD5();
spark3.append('Hi');

const state = spark3.getState();
const hashSoFar = spark3.end();
spark3.setState(state);

spark3.append(' there');
const hash3 = spark3.end(); 
console.log('hash3:', hash3);

On current master, output is:

hash1: d9385462d3deff78c352ebb3f941ce12
hash2: d9385462d3deff78c352ebb3f941ce12
hash3: 00000000000000000000000000000000

With this patch:

hash1: d9385462d3deff78c352ebb3f941ce12
hash2: d9385462d3deff78c352ebb3f941ce12
hash3: d9385462d3deff78c352ebb3f941ce12

I assume this is not intended behavior.

The problem is caused by .getState() copying the ._hash array by reference into the state object. The values in that array are then mutated by further operations, so contents of state object get changed too. My solution is simply to clone the array.

satazor commented 4 years ago

Thanks!

overlookmotel commented 4 years ago

Wow! That was fast!

Would you be able to release on NPM?

satazor commented 4 years ago

Released as v3.0.1.

overlookmotel commented 4 years ago

That was even faster! Huge thanks.