search

dfki-ric-underactuated-lab / double_pendulum

Dual purpose Acrobot and Pendubot Platform
BSD 3-Clause "New" or "Revised" License
33 stars 22 forks source link

HistorySAC #18

Closed HabibiM8 closed 1 week ago

HabibiM8 commented 1 week ago

Submission Simulation Stage for HistorySAC

fwiebe commented 1 week ago

Hi @HabibiM8 thanks for the pull request! I have the following remarks:

  1. Could you put the trained models in the folders (assuming you trained the policies with that design and model) data/policies/design_C.1/model_1.1/acrobot/history_sac data/policies/design_C.1/model_1.1/pendubot/history_sac
  2. Could you add short readme files for the leaderboards at leaderboard/acrobot/simulation_v2/readmes/history_sac.md leaderboard/acrobot/robustness_v2/readmes/history_sac.md leaderboard/pendubot/simulation_v2/readmes/history_sac.md leaderboard/pendubot/robustness_v2/readmes/history_sac.md
  3. If possible, could you also add the script(s) for training the models? You can put them in a folder here examples/reinforcement_learning/history_sac/
  4. When I try to load a trained model, I get the error 
    RuntimeError: Attempting to deserialize object on a CUDA device but torch.cuda.is_available() is False. If you are running on a CPU-only machine, please use torch.load with map_location=torch.device('cpu') to map your storages to the CPU.

    Would it be possible to add a check for whether a CUDA device is available or not and load the model accordingly? Also I get the warning that the behavior for torch.load may change in the future

    FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value),
which uses the default pickle module implicitly. It is possible to construct malicious pickle data
which will execute arbitrary code during unpickling
(See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details).
In a future release, the default value for `weights_only` will be flipped to `True`.
This limits the functions that could be executed during unpickling. Arbitrary objects will no
longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the
user via `torch.serialization.add_safe_globals`. We recommend you start setting
`weights_only=True` for any use case where you don't have full control of the loaded file.
Please open an issue on GitHub for any issues related to this experimental feature.