mailgun / expiringdict

Dictionary with auto-expiring values for caching purposes.
Apache License 2.0
347 stars 76 forks source link

Getting the length of the dictionary should return # of items not expired #39

Open chanchiem opened 5 years ago

chanchiem commented 5 years ago

Since items are only removed on retrieval, calling the len method on the dictionary will return all the items in the dictionary--including the expired items.

Length should iterate through the internal list and find what items are expired and remove them appropriately. Since it's internally implemented using an OrderedDict, we can perform a binary search to find what items in the dictionary have expired in Log(n) time

chanchiem commented 5 years ago

PR #40 fixes it. Doesn't use binary search approach yet. setitem cleans up implicitly, so amortized, it should be fast enough. If it makes a big performance impact, we know what algorithm would work best.

lucascott commented 4 years ago

I've noticed the bug still persists. The len() function applied to the ExpiringDict object doesn't return the expected value unless the same function is applied to the list of values first.

Code to reproduce the bug:

from expiringdict import ExpiringDict
import time
ed = ExpiringDict(max_len=8, max_age_seconds=2)
ed['a'] = 5
time.sleep(3)
print(len(ed))
print(len(ed.values()))
print(len(ed))

Output:

1
0
0
vshah-lifesize commented 4 years ago

@lucascott - Thats how its supposed to work, afaik. Expiring items are removed when the next mutating operation happens, so for a while they'll still claim memory.