JuliaData / MemPool.jl

High-performance parallel and distributed datastore for Julia
Other
23 stars 15 forks source link

Removing non-default directory during `exit_hook` #78

Closed StevenWhitaker closed 8 months ago

StevenWhitaker commented 9 months ago

https://github.com/JuliaData/MemPool.jl/blob/8508088b32caff629ce6905cb263f78cdc068cd6/src/MemPool.jl#L168

In exit_hook, the default_dir() is removed. What about if another directory is used, e.g., by passing diskpath to DiskCacheConfig, which in turn is passed to setup_global_device!? Does that custom diskpath remain, i.e., need to be manually deleted? And is this a bug or a feature?

jpsamaroo commented 8 months ago

So I think this logic is generally correct as-is if the default dir is used (since the default dir is unique for a given session, and is not created by the user), but we then have the question of whether MemPool should also clean up data in the dir used by setup_global_device! - I think by default the answer is "no", since we don't know the importance of that directory to the user. We already clean up the refs that MemPool swaps to disk during the session at exit, hopefully leaving the directory in the state it was prior to MemPool using it.

Do you have a case in mind where the current behavior is problematic?

StevenWhitaker commented 8 months ago

Oh, it's not problematic, I was just wondering if MemPool.jl was supposed to automatically delete the directory used by setup_global_device!. Your rationale sounds reasonable for why it shouldn't, so I'll just manually delete it in my case as needed. Thanks!