Reorganization into two frameworks, disagreement and multitask, plus a shared module for utils. Might merge a few things back if/when there are common classes and functions, which is not the case at the moment.
Changes model saving and loading - now requires environment variable CHECKPOINT_DIR, no longer set to save in local projects folder. Main reason is when running in cloud environments the local storage is not permanent and is lost when instance is terminated, hence checkpoint dir has to be set to a permanent storage outside the projects repository
Adds an ID to the project to avoid overriding
Adds gym to requirements
Fixes bug where when using backprop wasn't saving and loading models because agent had no attribute optimizer - now it checks whether it backprops through reward, and if positive, saves and load instead dynamics_optimizer and policy_optimizer
Change Readme to reflect latest changes
@vkakerbeck the commits are not super organized, but I suggest reviewing commit by commit. You can largely ignore the first which is about code reorganization, and modifies 70+ files, for that only the folder structure should be reviewed. The other commits contain more relevant code changes. I've tested it local, it seems to be saving; I didn't test loading.
I also modified the project name from robot_arm to disagreement, since it accepts other environments. I understand it is not only about disagreement, it can support curiosity based experiments, so if you disagree (pun intended) please feel free to suggest a more appropriate name.
optimizer- now it checks whether it backprops through reward, and if positive, saves and load insteaddynamics_optimizerandpolicy_optimizer@vkakerbeck the commits are not super organized, but I suggest reviewing commit by commit. You can largely ignore the first which is about code reorganization, and modifies 70+ files, for that only the folder structure should be reviewed. The other commits contain more relevant code changes. I've tested it local, it seems to be saving; I didn't test loading.
I also modified the project name from robot_arm to disagreement, since it accepts other environments. I understand it is not only about disagreement, it can support curiosity based experiments, so if you disagree (pun intended) please feel free to suggest a more appropriate name.