minerllabs / minerl

MineRL Competition for Sample Efficient Reinforcement Learning - Python Package
http://minerl.io/docs/
Other
711 stars 153 forks source link

Enable minimal CircleCI setup #552

Closed shwang closed 3 years ago

shwang commented 3 years ago

Configures a minimal CircleCI setup. I uploaded to DockerHub a minimal Docker image bundling the Ubuntu and Java dependencies that are required to pip install -e . MineRL. CircleCI pulls this Docker image and runs pytest on top of it.

Because the CircleCI setup is not complete, we should run both BuildKite and CircleCI for "hybrid" testing until we complete address the issues below:

This PR adds conditional skips to some data pytests when MINERL_DATA_ROOT is not set. This is because the minimal CircleCI set up doesn't download and cache any MineRL datasets right now. I'm leaving that for future work.

Other future work:

shwang commented 3 years ago

oof, it's failing on CircleCI's server (during MineRL Gradle build), but not on my local CircleCi runs. Maybe this is because we don't have enough memory in the free Runners offered to open source projects.

shwang commented 3 years ago
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 FAILURE: Build failed with an exception.
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * What went wrong:
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * Try:
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 Run with --info or --debug option to get more log output. Run with --scan to get full insights.
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * Exception is:
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 org.gradle.launcher.daemon.client.DaemonDisappearedException: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at org.gradle.launcher.daemon.client.DaemonClient.handleDaemonDisappearance(DaemonClient.java:242)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at org.gradle.launcher.daemon.client.DaemonClient.monitorBuild(DaemonClient.java:218)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at org.gradle.launcher.daemon.client.DaemonClient.executeBuild(DaemonClient.java:179)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at org.gradle.launcher.daemon.client.DaemonClient.execute(DaemonClient.java:142)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at 
[...]
org.gradle.wrapper.BootstrapMainStarter.start(BootstrapMainStarter.java:30)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at org.gradle.wrapper.WrapperExecutor.execute(WrapperExecutor.java:127)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526  at org.gradle.wrapper.GradleWrapperMain.main(GradleWrapperMain.java:61)
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 
ERROR    minerl.env.malmo.instance.41f52f:malmo.py:526 * Get more help at https://help.gradle.org
DEBUG    minerl.env._multiagent:_multiagent.py:660 Closing MineRL env...
shwang commented 3 years ago

OK, we are ready to push this -- we've got CircleCI running on tests that don't launch Minecraft, and it takes about 2.5 minutes to finish those tests. We can try launching Minecraft when we figure out memory requirements later.

brandonhoughton commented 3 years ago

WOO! This is great =)