Currently, only the SlurmEnvironment writes states to disk. We want to unify this and use this setup for all cases (#53).
As part of this change, we also want to turn the evaluator script into an executable script (#82, this also fixes #79).
The evaluation can then use resource limits and react correctly to the evaluator running out of resources (left over from #72).
With one run_dir for each run, it is also easier to keep track and report errors that should be investigated but do not terminate the search (e.g., a critical error in an evaluator).
The plan is to leave all run directories on the disk after the search and logging behavior becomes easier (related to #57, #77).
Currently, only the SlurmEnvironment writes states to disk. We want to unify this and use this setup for all cases (#53). As part of this change, we also want to turn the evaluator script into an executable script (#82, this also fixes #79). The evaluation can then use resource limits and react correctly to the evaluator running out of resources (left over from #72). With one run_dir for each run, it is also easier to keep track and report errors that should be investigated but do not terminate the search (e.g., a critical error in an evaluator). The plan is to leave all run directories on the disk after the search and logging behavior becomes easier (related to #57, #77).