search

ratt-ru / Stimela-classic

Containerized radio interferometry scripting framework -- NB: Classic version is no longer in active development, use stimela 2! See README for details.
GNU General Public License v2.0
28 stars 16 forks source link

set HOME, TMPDIR, TEMP, TMP sensibly inside container #674

Open o-smirnov opened 4 years ago

o-smirnov commented 4 years ago

The discussions in https://github.com/caracal-pipeline/caracal/issues/1200 and https://github.com/ratt-ru/ragavi/issues/78 highlight the fact that some packages are naughty, and like to open temp files in locations different to the nominal output directory -- which may be on a different filesystem even, thus causing strange IO errors such as "no space on device".

Stimela can be defensive about this: set HOME, TMPDIR, TEMP and TMP inside the container to the output directory, so that these problems are preempted.

Mulan-94 commented 4 years ago

~@o-smirnov, so I shouldn't change this in ragavi?~

gigjozsa commented 4 years ago

This would be great but it is hard to do it consistently as some software is even naughtier and hardcodes e.g. $HOME as /home/$USER, and then it would be great to provide some variables to redirect those directories (consistently) elsewhere (if for some reason those places are not the desired ones, if you e.g. want to share them across users), and put a big warning somewhere. Currently, for example in caratekit the user can temporarily redefine $HOME or some singularity folders, and that would be affected by a redirecting policy.

o-smirnov commented 4 years ago

Well, naughty software will always be cleverer than us, and this suggestion isn't a silver bullet. But just because it's not 100% foolproof doesn't mean it's not worth doing. It's simple and it protects from some naughtiness. For example, it will make us play nice with all software that relies on Python's tempfile module.