stanfordnmbl / osim-rl

Reinforcement learning environments with musculoskeletal models
http://osim-rl.stanford.edu/
MIT License
882 stars 249 forks source link

random_seed is not set in def change_model() of class ProstheticsEnv(OsimEnv) #158

Closed huschen closed 6 years ago

huschen commented 6 years ago

Though stated in the document of interface.

seed - starting seed for the random number generator. If the seed is None, generation from the previous seed is continued.

and here is the source code:

def change_model(self, model='3D', prosthetic=True, difficulty=0, seed=None):
    if (self.model, self.prosthetic) != (model, prosthetic):
        self.model, self.prosthetic = model, prosthetic
        self.load_model(self.model_paths[self.get_model_key()])

seed is not set.

kidzik commented 6 years ago

Thanks for catching that. There is no stochasticity in the first round, but there will be in the second round https://github.com/stanfordnmbl/osim-rl/issues/159 and the seed will become relevant

huschen commented 6 years ago

I am trying to see if it is possible to make the training deterministic. Is there any non-deterministic introduced by OpenSim environment? Any known computations (besides GPU computation)? Any other random seeds to be set besides the following?

 tf.set_random_seed(x)
 np.random.seed(x)
 random.seed(x)

Cheers.

kidzik commented 6 years ago

I suppose there might be potentially some stochasticity in sampling for the integrator. @carmichaelong is there any stochasticity on the OpenSim side during forwarding simulations?

carmichaelong commented 6 years ago

There shouldn't be any stochastic events during integrations.

kidzik commented 6 years ago

Thanks, @carmichael. @huschen closing, for now, let us know if you experience any nondeterministic behavior.