Currently ThreadPoolEngine and ProcessPoolEngine now pass the working_dir: default tasks_working_dir.
Executing functions on these engines affect subsequent function execution when they say change directory for eg, unlike GlobusComputeEngine. When executing functions, our execute_task wrapper currently attempts to look for a tasks_working_dir in the current dir, creates it if not present, and then switches into it. Unfortunately, each subsequent function finds itself one directory deeper.
This PR updates GlobusComputeEngineBase to check if the working_dir is an absolute path, and if not make it absolute. This ensures that every function would run in ~/.globus_compute/<endpoint_name>/tasks_working_dir.
working_dir is now a configurable option for all engines.
tests to confirm that running multiple functions do not change directory
Fixes # (issue)
[sc-36710]
Type of change
Choose which options apply, and delete the ones which do not apply.
Description
Currently
ThreadPoolEngineandProcessPoolEnginenow pass the working_dir: defaulttasks_working_dir. Executing functions on these engines affect subsequent function execution when they say change directory for eg, unlikeGlobusComputeEngine. When executing functions, ourexecute_taskwrapper currently attempts to look for atasks_working_dirin the current dir, creates it if not present, and then switches into it. Unfortunately, each subsequent function finds itself one directory deeper.GlobusComputeEngineBaseto check if theworking_diris an absolute path, and if not make it absolute. This ensures that every function would run in~/.globus_compute/<endpoint_name>/tasks_working_dir.working_diris now a configurable option for all engines.Fixes # (issue)
[sc-36710]
Type of change
Choose which options apply, and delete the ones which do not apply.