search

stanfordnmbl / osim-rl

Reinforcement learning environments with musculoskeletal models
http://osim-rl.stanford.edu/
MIT License
882 stars 249 forks source link

Dimensions of observation changed between round 1 & round 2 with difficulty set to 0 #167

Closed rwightman closed 5 years ago

rwightman commented 6 years ago

The observation vector size changed from 158 to 160 even with difficulty set to 0 with the new round 2 changes. The change to 3D COM was the cause of this https://github.com/stanfordnmbl/osim-rl/pull/165/commits/8512606186da97daa1b94fbc54ccabbaf4c4bda8 and I don't believe this change was pulled into the repo being used by all until now, even though it was made a while ago? At least I didn't have it when following the original directions.

It would seem that the COM addition to observation vec needs to be wrapped in difficulty check if models trained on round 1 are still to be run/submitted with the newly deployed code

Ex:

cm_pos = [state_desc["misc"]["mass_center_pos"][i] - pelvis[i] for i in range(2)]
cm_vel = state_desc["misc"]["mass_center_vel"]
cm_acc = state_desc["misc"]["mass_center_acc"]
if self.difficulty == 0:
    cm_vel = cm_vel[0:2]
    cm_acc = cm_acc[0:2]
res += cm_pos + cm_vel + cm_acc
rwightman commented 6 years ago

Also, as you can see above, if the intent of the 3d COM was to have them pass through as 3d to the projected obs vector for pos, vel, and acc in the ProstheticsEnv, the pos is still being cut to 2d by the range(2) above for all scenarios.

kidzik commented 6 years ago

Good point, thank you! Indeed the change was merged with the master branch just this week. This problem has been mentioned in one of the previous issues and we wanted to fix that for the Round 2 (https://github.com/stanfordnmbl/osim-rl/issues/129).

We want to keep almost the same interface for the first and the second round for convenience. As you mentioned, it is possible to map the current observation to the old one. We return a dictionary on the server side so the code prepared for the old version should work.