search

beyretb / AnimalAI-Olympics

Code repository for the Animal AI Olympics competition
Apache License 2.0
572 stars 84 forks source link

Question about example config files #76

Closed seann999 closed 4 years ago

seann999 commented 4 years ago

I'm not sure if this is the place to ask this, but how much do the example config yaml files reflect the actual evaluation environments? I ask this because even an agent with 99.7% success rate in 1-Food.yaml yields only about 20/30 on eval.ai. It's hard to think of any modifications to make it more difficult assuming the environment has no obstacles and one food.

seann999 commented 4 years ago

Okay, I'm guessing I should read the descriptions of each category more carefully: "Allowed objects: All goals, Nothing else" So I guess Category 1 can include moving goals and negative goals (all 4 types).

mdcrosby commented 4 years ago

Correct.