awolverp
commented
5 months ago Hi, sorry for the delay. Cachebox uses the Rust memory allocator. It doesn't serialize key values; instead, it keeps them in a HashMap because serializing objects would slow down the process.
All implementations use a fast HashMap that uses the Rust memory allocator, and the HashMap needs to use heap memory. We don't need GC here.
When a cache needs to be deallocated, it needs to call the garbage collector for Python objects that are in the HashMap.
So only the keys and values in caches need to use the garbage collector for their reference counting.
By lack of email or discussion board, I am asking this question there (sorry for the noise) : is the cache off-heap? I mean, does it serialize the keys and values (pickle) and store them only through the rust memory allocator? Or does it keep the python objects for the keys and values around, keeping pressure on the Python garbave collector?
Because, in the first case, it would be an awesome selling point for your library, as an off-heap dictionary. This is a very common need in the old Java world, but strangely no quaity Python libraries exists to store large dictionaries off-heap (aside from Sqlite & full-fledged in-process databases off course). The Python garbage collector being not-so-great, large dictionaries in the Python heap are cumbersome.
I scanned quickly your source code but could not find a clear answer to this.