stanfordnmbl / osim-rl

Reinforcement learning environments with musculoskeletal models
http://osim-rl.stanford.edu/
MIT License
882 stars 249 forks source link

equilibrateMuscles() not called in osim.py? #133

Closed fwillett closed 5 years ago

fwillett commented 6 years ago

I think that equilibrateMuscles() should be called in osim.py when initializing the simulation? I noticed it wasn't being called and, when I added it, it helped some of our reaching simulations a lot by fixing bad startup behavior in the muscles.

I think it needs to be called in OsimModel's init function like this (at the end):

    state = self.model.initSystem()
    self.model.equilibrateMuscles(state)

and in reset() like this (at the beginning):

    self.state = self.model.initializeState()
    self.model.equilibrateMuscles(self.state)
kidzik commented 6 years ago

Thanks! Indeed that might affect the starting point, we will revise that in the upcoming release.

adwardlee commented 6 years ago

Is it necessary to add this function? I think it was not used in the competition last year either?

carmichaelong commented 6 years ago

It wasn't used in last year's version, but it's more important this year because the dictionary has information about the muscle states, such as fiber length and force, which may change greatly during startup due to equilibrateMuscles(). Furthermore, it brings the first step in the simulation more in line with the rest of the simulation, since equilibrateMuscles() is inherently called with each integration step.

AdamStelmaszczyk commented 6 years ago

@kidzik When this will be fixed?

kidzik commented 6 years ago

This is a minor issue (affecting only the first step of the simulation) which will be fixed most likely in the second round.

AdamStelmaszczyk commented 6 years ago

It's not that minor as it seems, many algorithms calculate mean and standard deviation of the observations to normalize the input. It's broken when the values in the first step are huge.

kidzik commented 5 years ago

We run equilibrateMuscles now in the reset function https://github.com/stanfordnmbl/osim-rl/blob/master/osim/env/osim.py#L237