Yiling-J / theine

high performance in-memory cache
BSD 3-Clause "New" or "Revised" License
364 stars 5 forks source link

Memory Leak with Thread creation #18

Closed Effervex closed 12 months ago

Effervex commented 12 months ago

When a new Cache is initialised, a new Thread is spawned from the following code in the init method:

self._maintainer = Thread(target=self.maintenance, daemon=True)
self._maintainer.start()

The Thread is never stopped and even if the Cache dies the thread remains and holds onto the cached data. Even if the data is cleared, the Thread will continue to persist.

Minimal Reproducible Example

from theine import Cache

# Loop endlessly - this will crash after a few minutes
while True:
    # Create a new cache - which creates a new Thread
    c = Cache("tlfu", 10000)
    # Add some fake data. Not necessary for the crash, but it takes up memory
    c.set("data", ["data"] * 2048)
    # Remove the cache, just to be explicit.
    del c
Yiling-J commented 12 months ago

@Effervex Thanks for reporting, I will add a close method to cache instance, and override the __del__ to make sure the extra thread is also stopped

Yiling-J commented 12 months ago

@Effervex 0.4.1 released, call close API explicitly if you want to stop the thread immediately. I also override the __del__ method, but seems cache instance won't be removed by gc immediately if you just del it. To remove the cache entirety, please call cache.clear and cache.stop

Effervex commented 12 months ago

call cache.clear and cache.stop

Do you mean cache.clear and cache.close? cache.stop doesn't exist.

Yiling-J commented 12 months ago

@Effervex my mistake, if you want to delete a cache instance, first call cache.clear and cache.close, then del cache

Effervex commented 12 months ago

Got it. I've tested it and it seems to work as expected. Thanks for the quick fix!