Issue SMAC algorithm

[ijmarrero]

I am having code issues with SMAC implementation (pull request #137)

Traceback (most recent call last): File "examples/smac/generate_ant_data.py", line 74, in main() File "examples/smac/generate_ant_data.py", line 70, in main use_gpu=gpu, File "/home/ubuntu/rlkit-master/rlkit/launchers/launcher_util.py", line 605, in run_experiment run_experiment_kwargs File "/home/ubuntu/rlkit-master/rlkit/launchers/launcher_util.py", line 174, in run_experiment_here return experiment_function(raw_variant) File "/home/ubuntu/rlkit-master/rlkit/torch/smac/pearl_launcher.py", line 173, in pearl_experiment algorithm.train() File "/home/ubuntu/rlkit-master/rlkit/core/meta_rl_algorithm.py", line 303, in train self.enc_replay_buffer.task_buffers[task_idx].clear() AttributeError: 'SimpleReplayBuffer' object has no attribute 'clear'

and

Traceback (most recent call last): File "examples/smac/generate_ant_data.py", line 74, in main() File "examples/smac/generate_ant_data.py", line 70, in main use_gpu=gpu, File "/home/ubuntu/rlkit-master/rlkit/launchers/launcher_util.py", line 605, in run_experiment run_experiment_kwargs File "/home/ubuntu/rlkit-master/rlkit/launchers/launcher_util.py", line 174, in run_experiment_here return experiment_function(raw_variant) File "/home/ubuntu/rlkit-master/rlkit/torch/smac/pearl_launcher.py", line 173, in pearl_experiment algorithm.train() File "/home/ubuntu/rlkit-master/rlkit/core/meta_rl_algorithm.py", line 436, in train self.trainer.train(batch) File "/home/ubuntu/rlkit-master/rlkit/torch/torch_rl_algorithm.py", line 40, in train self.train_from_torch(batch) File "/home/ubuntu/rlkit-master/rlkit/torch/smac/pearl.py", line 184, in train_from_torch action_distrib.rsample_logprob_and_pretanh() AttributeError: 'TanhNormal' object has no attribute 'rsample_logprob_and_pretanh'

[vitchyr]

@ijmarrero Thanks for bringing up this issue. I must not have caught this when merging this code in. I'll work on testing this further, but it may take me a while. In the meantime:

For the first issue, please add this to rlkit/data_management/simple_replay_buffer.py: SimpleReplayBuffer

    def clear(self):
        self._top = 0
        self._size = 0
        self._episode_starts = []
        self._cur_episode_start = 0

For the second issue, please add this to rlkit.torch.distributions.py:TanhNormal

    def rsample_logprob_and_pretanh(self):
        value, pre_tanh_value = self.rsample_with_pretanh()
        log_p = self.log_prob(value, pre_tanh_value)
        return value, log_p, pre_tanh_value

[ijmarrero]

Thanks @vitchyr

I have just fixed the code but I new issues arise.

Traceback (most recent call last): File "examples/smac/generate_ant_data.py", line 74, in main() File "examples/smac/generate_ant_data.py", line 70, in main use_gpu=gpu, File "/home/ubuntu/rlkit-master/rlkit/launchers/launcher_util.py", line 605, in run_experiment run_experiment_kwargs File "/home/ubuntu/rlkit-master/rlkit/launchers/launcher_util.py", line 174, in run_experiment_here return experiment_function(raw_variant) File "/home/ubuntu/rlkit-master/rlkit/torch/smac/pearl_launcher.py", line 173, in pearl_experiment algorithm.train() File "/home/ubuntu/rlkit-master/rlkit/core/meta_rl_algorithm.py", line 436, in train self.trainer.train(batch) File "/home/ubuntu/rlkit-master/rlkit/torch/torch_rl_algorithm.py", line 40, in train self.train_from_torch(batch) File "/home/ubuntu/rlkit-master/rlkit/torch/smac/pearl.py", line 188, in train_from_torch policy_log_std = action_distrib.log_std AttributeError: 'TanhNormal' object has no attribute 'log_std'

[vitchyr]

@ijmarrero this should be fixed with #147 . Let me know if you have any other issues

[ijmarrero]

Hi Vitchyr,

Thanks a lot!. I am running it again

Best

El mar, 12 oct 2021 a las 0:57, Vitchyr Pong @.***>) escribió:

@ijmarrero https://github.com/ijmarrero this should be fixed with #147 https://github.com/rail-berkeley/rlkit/pull/147 . Let me know if you have any other issues

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rail-berkeley/rlkit/issues/146#issuecomment-940502340, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVYYF63JZN5IAIOOS7AZTDTUGNTVLANCNFSM5EV2WDQA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.