JuliaCollections / LRUCache.jl

An implementation of an LRU Cache in Julia
Other
56 stars 23 forks source link

Serializing Large LRUs hangs/StackOverflows #45

Closed jarbus closed 9 months ago

jarbus commented 9 months ago

The following code hangs on my machine and performs a stackoverflow on another:

using LRUCache
using Serialization

# Create an LRU cache with a capacity of 100k elements
cache = LRU{Int, Int}(maxsize=100_000)

# Populate the cache with dummy data
for i in 1:100_000
    cache[i] = i
end

serialize("cache.jls", cache)

I think this is because when there are too many linked nodes, the stack overflows before all the nodes can be serialized. I'm going to be writing some custom serialization logic to handle large LRUs, was wondering if upstream might want the changes.

Jutho commented 9 months ago

I think the linked lists will indeed need custom serialisation, as otherwise probably the standard approach tries to traverse from every node down as if they were all infinitely deep trees of independent nodes.

jarbus commented 9 months ago

@Jutho would you be interested in a PR?

Jutho commented 9 months ago

Sure, why not. We have so far not had the need to serialize LRUs, but other people might need it, and so might we at some point.

jarbus commented 9 months ago

Submitted a PR, #46. Let me know what you think.