search

Unity-Technologies / obstacle-tower-env

Obstacle Tower Environment
Apache License 2.0
541 stars 124 forks source link

Some seeds are jumping levels.... #99

Closed Sohojoe closed 5 years ago

Sohojoe commented 5 years ago

Some seeds skip levels:

see video - https://twitter.com/iAmVidyaGamer/status/1136867273057030144

so far I have found the following

Seed 5 - jumps from level 2 to level 4 Seed 1 - jumps from level 3 to level 5 Seed 1 - jumps from level 9 to level 11 Seed 38 - jumps from level 2 to level 4 Seed 38 - jumps from level 9 to level 11 Seed 52 - jumps from level 9 to level 11

awjuliani commented 5 years ago

Thanks for pointing this out @Sohojoe. It is indeed a serious bug, and likely impacting the scores on the leaderboard. I looked into it and have found the cause and made a fix. Once it has been reviewed, we will roll out a new version of the tower which addresses the issue.

I appreciate your persistence in tracking down the cause of the strange behavior you were seeing. Sending the specific seeds and floor numbers was especially helpful in allowing me to quickly replicate the issue.

Sohojoe commented 5 years ago

@awjuliani - that's great to hear there is a fix so soon! I would never have thought to look into this had we not had the exchange on twitter!

awjuliani commented 5 years ago

We've just pushed v2.2, which should address this issue. It may take a few hours for AICrowd to correspondingly update their evaluator build however, just fyi.