Summary: Migrated Will's refactor of the agent (both HPO and tune).
Demo:
All eight commands in pat_test.sh should work out of the box on any machine.
Details:
From Will's refactor, the param space for HPO is now completely specified in the _build_space() function as opposed to being specified in multiple different code functions, one config file, and some CLI args.
From Will's refactor, instrumentation is now done using @time_record.
Adding on top of Will's refactor, all input and output paths in all stages are now stored in the correct places in dbgym_workspace/. Files automatically "flow" from one stage to the next by default, without the user needing to specify these paths. All paths are portable as well and work on any machine. Finally, all files that are opened are saved in the run_*/ directory for reproducibility.
Adding on top of Will's refactor, Postgres is now handled in a unified way across all stages of the embedding using the functions in pg.py and the pgdata.tgz file generated by python task.py dbms postgres pgdata .... Cleanup of pgdata/ports are also handled in a way that avoids leaking files. In pg_conn.py, the three different pgdatas (the "pristine" pgdata, the "checkpoint" pgdata, and the "temporary checkpoint" pgdata) are properly defined and separated now (whereas "pristine" and "checkpoint" were the same file earlier).
Summary: Migrated Will's refactor of the agent (both HPO and tune).
Demo: All eight commands in
pat_test.shshould work out of the box on any machine.Details:
_build_space()function as opposed to being specified in multiple different code functions, one config file, and some CLI args.@time_record.dbgym_workspace/. Files automatically "flow" from one stage to the next by default, without the user needing to specify these paths. All paths are portable as well and work on any machine. Finally, all files that are opened are saved in therun_*/directory for reproducibility.pg.pyand thepgdata.tgzfile generated bypython task.py dbms postgres pgdata .... Cleanup of pgdata/ports are also handled in a way that avoids leaking files. Inpg_conn.py, the three different pgdatas (the "pristine" pgdata, the "checkpoint" pgdata, and the "temporary checkpoint" pgdata) are properly defined and separated now (whereas "pristine" and "checkpoint" were the same file earlier).