intel / isa-l_crypto

Other
275 stars 80 forks source link

Rolling hash how to roll out a single byte? #90

Closed leiless closed 2 years ago

leiless commented 2 years ago

Hi, @gbtucker.

How can I roll a single byte out via rolling_hash2_run()? it is possible? (since it's a cryptographic rolling hash)

My goal is to replace the Adler-32 rolling checksum algorithm with the ISAL crypto rolling hash. I can implement update() simply by making mask = 1, trigger = 0, since hash | mask = hash | 1 = 0 is always false, so the whole buffer was calculated, so I can utilize the 64-bit hash value.

And roll a single byte in can apply the same manner aforementioned.

(P.S. roll-in = add, roll-out = remove)

But I don't how to roll a single byte out of the window array.

I want to do something that is similar to

// Function signatures
int sum(byte[]|Buffer data, [int current_sum])
int roll(int sum, int sum_length, byte removed_byte, [byte added_byte])

var sum = adler32.sum(data.slice(0, 5));              // Again, start with the beginning 5 byte chunk of the buffer.
sum = adler32.roll(sum, 5, data[0], data[5]);         // Move the sum forward one byte.
sum = adler32.roll(sum, 5, data[1], data[6]);         // Move the sum forward another byte.

var sum = adler32.sum(data);                       // Start with the sum of the entire buffer.
sum = adler32.roll(sum, data.length, data[0]);     // Move the sum forward one byte, removing the first byte. There is no next byte to add so the method is only given 3 arguments (null would also work). Note that the length argument is the length of the buffer used to calculate the original sum.
sum = adler32.roll(sum, data.length - 1, data[1]); // Move the sum forward again. Note that the length is of the buffer represented by the previous sum.

assert.deepEqual(sum, adler32.sum(data.slice(2))); // You should end up with the sum of the buffer minus the first two bytes.

https://github.com/Shakeskeyboarde/adler32#examples

With current rolling_hash2_run() implementation, I can only roll-in bytes, but I cannot do something like adler32.roll(sum, 5, the_removed_byte) to kick a single byte out.

gbtucker commented 2 years ago

But I don't how to roll a single byte out of the window array.

The function rolling_hash2_run() does this automatically. It rolls in each byte at the beginning of the window and rolls out each byte at the end of the window while also checking (hash & mask) == trigger. You could do one byte at a time by setting the max_len = 1 but it is more efficient to run for longer stretches.

leiless commented 2 years ago

It rolls in each byte at the beginning of the window and rolls out each byte at the end of the window while also checking (hash & mask) == trigger.

rolling_hash2_run() seems do roll out and roll in at the same time even if max_len = 1, i.e. rotate. But I need only to roll out a single byte out the leftmost of the sliding window.

leiless commented 2 years ago

It rolls in each byte at the beginning of the window and rolls out each byte at the end of the window while also checking (hash & mask) == trigger.

rolling_hash2_run() seems do roll out and roll in at the same time even if max_len = 1, i.e. rotate. But I need only to roll out a single byte out the leftmost of the sliding window.

Hi, sorry to bother you @gbtucker, any update on this?

gbtucker commented 2 years ago

It rolls in each byte at the beginning of the window and rolls out each byte at the end of the window while also checking (hash & mask) == trigger.

rolling_hash2_run() seems do roll out and roll in at the same time even if max_len = 1, i.e. rotate. But I need only to roll out a single byte out the leftmost of the sliding window.

No, sorry but the three operations (add new contribution, subtract contribution from end of window, and check trigger) are together in one call so you cannot just do one of the three within this API.

leiless commented 2 years ago

Okay, thanks for your reply.