ascv / HyperLogLog

Fast HyperLogLog for Python.
MIT License
99 stars 19 forks source link

Return true if registers changed on add method #34

Closed mayankasthana closed 4 years ago

mayankasthana commented 4 years ago

In hll.add(data), if the HLL internal register was altered we can return True otherwise False.

My usecase is that I have a high throughput stream processing application, where I need to lookup and update multiple HLLs to an external store. The data in the stream is quite repetitive, so the HLL may not be updated most of the time.

If the HLL is not updated, then we don't need to update it to the external store. Currently, to detect if the HLL has changed, we have to store the registers, add data to the HLL and then compare the registers again with the old value. This is an expensive operation and is not needed if hll.add(data) itself returns whether any register has been updated or not.

Redis' PFADD command also returns 1 or 0 depending on whether the HLL internal register was altered or not.

I can send a PR if interested.

mayankasthana commented 4 years ago

Now that I think about it, merge could also easily return whether the registers were updated or not.

ascv commented 4 years ago

Returning a boolean indicator for merge would be fine to. I'm closing since the original issue has been resolved.