search

marius311 / Memoization.jl

Easily and efficiently memoize any function, closure, or callable object in Julia.
MIT License
95 stars 2 forks source link

Making Memoization.jl threadsafe. #7

Open krishvishal opened 3 years ago

krishvishal commented 3 years ago

Can the following library be used to make Memoization.jl threadsafe?

ThreadSafe Dict: https://github.com/wherrera10/ThreadSafeDicts.jl

marius311 commented 3 years ago

Nice! Didn't know about it. I think that should work for memoized top-level functions. At least, it will if the answer to this is yes (which I think it is, just wanting to verify).

I don't think it will currently work for memoized callables/closures though, since looking up the right cache itself is still happening with the non-threadsafe Memoization.cache :: IdDict. To make those threadsafe, I need to add some locking to this package (I don't think we could use ThreadsafeDict there, since the underlying thing needs to be an IdDict, which that doesn't seem to support yet). If this is important to you now, I'm happy to take a PR if you can show the performance hit is not too bad. Otherwise I will play around with this myself at some point in the future.

marius311 commented 3 years ago

Well, the answer wasn't exactly yes but it should still work,

using ThreadSafeDicts, Memoization
@memoize ThreadSafeDict foo(x) = ...