search

stanfordnmbl / osim-rl

Reinforcement learning environments with musculoskeletal models
http://osim-rl.stanford.edu/
MIT License
893 stars 249 forks source link

Update on leaderboard after the new version 1.5 #85

Closed LinZichuan closed 6 years ago

LinZichuan commented 7 years ago

I am confused whether the submissions before version 1.5 on the learderboard are still valid.

AdamStelmaszczyk commented 7 years ago

Yep, now no participant knows which are valid and which are not.

Related discussion in the end of https://github.com/stanfordnmbl/osim-rl/issues/64.

kidzik commented 7 years ago

Right, that is an unfortunate consequence of the last bugfix (we can't check retrospectively which of the solutions were submitted with [0,1] activations. Therefore, we increased the number of spots in the second round (we planned 10 spots, now it's everyone with at least 15 points). The second round will be run in the 1.5 environments.

AdamStelmaszczyk commented 7 years ago

Oh, I thought you were persisting the actions into some database. Thought that it was useful for generating visualizations and reproducing problems (e.g. the one when OpenSim gets very slow on some steps or submit failures/continuations). Possibly a good thing to have in the future.

kidzik commented 7 years ago

We have them for most of the submissions but not all of them. At the beginning it was used only for recreating the videos -- we haven't anticipated how useful it might be in other scenarios. Now we know :)