search

allenai / oocmap

A file-backed dictionary for Python
Apache License 2.0
11 stars 0 forks source link

Garbage Collection #4

Open dirkgr opened 3 years ago

dirkgr commented 3 years ago

You can delete items from an OOCMap, but they will not be garbage collected.

GC doesn't have to happen live as soon as something can be deleted. We can just do a mark-and-sweep every once in a while, maybe just in response to a direct call to OOCMap.gc().

dirkgr commented 3 years ago

Actually, this is a little more complicated than that. Even if items are no longer referenced by top-level OOCMap entries, they might be referenced by Python objects that are still floating around. Without a reference count that's stored in the map (very expensive!), we can't prevent that.

GC will have to be an offline operation that is only done when you can be quite sure that nobody is using the file at the same time.

dirkgr commented 3 years ago

they might be referenced by Python objects that are still floating around.

Maybe we can just throw a ConcurrentModificationError in those cases.