populse / capsul

Collaborative Analysis Platform : Simple, Unifying, Lean
Other
7 stars 14 forks source link

Working directory of processes & how to handle relative file paths #305

Open ylep opened 9 months ago

ylep commented 9 months ago

When a relative path is passed to a File parameter of a process, it should probably be converted to an absolute path before going to the execution engine.

A related issue is: if a process writes files to its current directory, what do these files become once execution is finished? We should probably force the working directory of workers to be in a temporary directory that is deleted after execution

sapetnioc commented 8 months ago

In order to be as close as what a local Python script does with relative path, it is necessary to convert relative paths to absolute paths on the client side because this is where the current directory may have a meaning. If we use controllers callbacks to check all path parameters modification we would slow down all processes for a very rare event. Therefore, resolving relative path on process start seems to be the good moment.

For the second point, I think we cannot be protected from a process doing scrap. It can write to current directory, to /tmp or any directory and leave files here. I am not in favor of creating a temporary directory for every job just to protect the very few of them that could write in current directory. But if it is necessary, it should be allowed to enable/disable it in config.