When running search on many targets using preemptible/unreliable compute, it may happen that the run fails having completed a fraction of targets. If such a run is restarted, it will generate a new timestamped output directory and start from scratch. This PR adds a flag append_timestamp_to_dir, which when set to False turns off timestamping, meaning that a restarted run would use the same directory as the old one. On top of this, it implements logic to skip targets which were completed earlier; a target that was partially completed (for example the search statistics were saved but it crashed when dumping the graph) will be purged and recomputed. Finally, the PR also includes a few small tweaks to the surrounding code: adding guards around visualization imports (to make sure search can be ran with the base environment without graphviz) and deleting an __init__.py file in an old empty directory.
When running search on many targets using preemptible/unreliable compute, it may happen that the run fails having completed a fraction of targets. If such a run is restarted, it will generate a new timestamped output directory and start from scratch. This PR adds a flag
append_timestamp_to_dir, which when set toFalseturns off timestamping, meaning that a restarted run would use the same directory as the old one. On top of this, it implements logic to skip targets which were completed earlier; a target that was partially completed (for example the search statistics were saved but it crashed when dumping the graph) will be purged and recomputed. Finally, the PR also includes a few small tweaks to the surrounding code: adding guards around visualization imports (to make sure search can be ran with the base environment withoutgraphviz) and deleting an__init__.pyfile in an old empty directory.