search

rail-berkeley / rlkit

Collection of reinforcement learning algorithms
MIT License
2.45k stars 550 forks source link

Issue with reproducibility #102

Closed RushikeshJoshi4 closed 4 years ago

RushikeshJoshi4 commented 4 years ago

I'm trying to train a model such that it can be reproduced again. I tried setting the seed parameter in the run_experiment function, however, I am not able to get the same result again. Is there something I'm missing here?

More details: Training SAC according to the examples/sac.py Using mujoco environments. I am currently working on the Humanoid v3. Everything else is quite similar the the examples/sac.py file.

vitchyr commented 4 years ago

I realized that I don't set the random seed for pytorch, so perhaps that's why you're getting inconsistent result. Perhaps setting the seed there will help get consistent results.

RushikeshJoshi4 commented 4 years ago

Sorry for the delayed reply. Yes Vitchyr, you're right. Just for reference of those are trying to create reproducible results:

  1. Make sure set torch.manual_seed() where the seed is set for numpy and random.
  2. Also, make sure you set the seeds appropriately for the environments too. Otherwise, the environment will give you random observations. But you can't do this just after you create the environment. Otherwise, every time the environment is reset (in the rollout function), it will give you same observations. Instead you want to set the environment seeds differently for different replay of each of different epochs. So, make sure that is set appropriately.

Closing this issue since I realised my mistake.