Guidance on avoiding race conditions when multiple jobs try to access and (re)generate the same scratch space

JuliaPackaging / Scratch.jl

Scratch spaces for all your persistent mutable data needs

MIT License

58 stars 11 forks source link

Guidance on avoiding race conditions when multiple jobs try to access and (re)generate the same scratch space #26

Open fingolfin opened 2 years ago

fingolfin commented 2 years ago

Motivated by https://github.com/oscar-system/Polymake.jl/issues/381 :

A user may do @everywhere using PACKAGENAME on a package which uses scratch spaces. if the scratch space is missing/outdated, then each of the parallel jobs may try to regenerate it, creating a mess.

I think it would be good to warn about this situation in the Scratch.jl documentation, and perhaps also provide some guidance on how to deal with that.

fredrikekre commented 2 years ago

https://github.com/vtjnash/Pidfile.jl maybe?

fingolfin commented 2 years ago

Perhaps? The documentation of Pidfile.jl is a bit sparse... I assume one ought to use one of its API functions to create a lock file, then create the scratch space and finally release the lock? An example would be useful. I also wonder how well it e.g. works on NFS volumes (still see those in labs for home dirs), and whether it works on Windows.