flier / pyfasthash

Python Non-cryptographic Hash Library
Apache License 2.0
281 stars 52 forks source link

Fix memory leak on bufferable objects #58

Closed marcoffee closed 1 year ago

marcoffee commented 2 years ago

This pull request addresses the issue #57. Note that, the bug reported happens for any bufferable object, except for bytes, which are handled separately on the code.

Test code:

import pyhash
import psutil
import numpy as np

hasher = pyhash.highway_128()
memory_before = psutil.Process().memory_info().rss

for _ in range(10):
    hasher(np.random.randint(0, 255, 2 ** 30, np.uint8))

memory_after = psutil.Process().memory_info().rss

print("memory increase =", memory_after - memory_before)

current version: memory increase = 10737897472 after correction: memory increase = 593920 (probably some objects waiting for garbage collection)

flier commented 1 year ago

Thanks