Run ai2thor headless on headless linux server couldn't get normal event.frame？

[li3cmz]

I run ai2thor (with unity command in graphics mode and nographics mode ) on headless linux server using the docker provided. The problem I meet is as follow:

First, if I don't change any code and run the code on docker below: test.py

c = ai2thor.controller.Controller() c.docker_enabled = True c.start()

Then, I will get a error in docker container build from ai2thor/ai2thor-nvidia-version:tag:

Unable to preload the following plugins:ScreenSelector.so

The complete log is as follow:

~/.ai2thor/releases/thor-201810021256-Linux64# ./thor-201810021256-Linux64 error: XDG_RUNTIME_DIR not set in the environment. Set current directory to /root/.ai2thor/releases/thor-201810021256-Linux64 Found path: /root/.ai2thor/releases/thor-201810021256-Linux64/thor-201810021256-Linux64 Mono path[0] = '/root/.ai2thor/releases/thor-201810021256-Linux64/thor-201810021256-Linux64_Data/Managed' Mono config path = '/root/.ai2thor/releases/thor-201810021256-Linux64/thor-201810021256-Linux64_Data/Mono/etc' Unable to preload the following plugins: ScreenSelector.so PlayerPrefs - Creating folder: /root/.config/unity3d/Allen Institute for Artificial Intelligence PlayerPrefs - Creating folder: /root/.config/unity3d/Allen Institute for Artificial Intelligence/AI2-Thor Logging to /root/.config/unity3d/Allen Institute for Artificial Intelligence/AI2-Thor/Player.log error: XDG_RUNTIME_DIR not set in the environment. Stacktrace:

Native stacktrace:

/root/.ai2thor/releases/thor-201810021256-Linux64/thor-201810021256-Linux64_Data/Mono/x86_64/libmono.so(+0x984ea) [0x7fe7632024ea] /lib/x86_64-linux-gnu/libpthread.so.0(+0x1138f) [0x7fe766a0e38f] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38) [0x7fe765bbf428] /lib/x86_64-linux-gnu/libc.so.6(abort+0x169) [0x7fe765bc1029] ./thor-201810021256-Linux64() [0x9e03bb] /root/.ai2thor/releases/thor-201810021256-Linux64/thor-201810021256-Linux64_Data/Mono/x86_64/libmono.so(+0xd8d20) [0x7fe763242d20] /root/.ai2thor/releases/thor-201810021256-Linux64/thor-201810021256-Linux64_Data/Mono/x86_64/libmono.so(+0x371ee) [0x7fe7631a11ee] /lib/x86_64-linux-gnu/libpthread.so.0(+0x1138f) [0x7fe766a0e38f] ./thor-201810021256-Linux64(SDL_GL_GetAttribute_REAL+0x1e3) [0x145e0e2] ./thor-201810021256-Linux64() [0x12c0dd4] ./thor-201810021256-Linux64() [0x12b860d] ./thor-201810021256-Linux64() [0x12b869f] ./thor-201810021256-Linux64() [0x1289351] ./thor-201810021256-Linux64() [0x460cff] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xef) [0x7fe765baa82f] ./thor-201810021256-Linux64() [0x46e038]

Debug info from gdb:

Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application.

Aborted (core dumped)

Second, I change the code below:

In file "anaconda2/lib/python2.7/site-packages/ai2thor-0.0.44-py2.7.egg/ai2thor/controller.py" line 598 fuction: def unity_command(self, width, height)

command += " -screen-fullscreen %s -screen-quality %s -screen-width %s -screen-height %s -batchmode -nographics" % (fullscreen, QUALITY_SETTINGS[self.quality], wi dth, height)

Then rerun the test.py, and no doubtly，the application can run successfully. But all the value of RGB I get from event.frame is 205. That it to say, unable to get the program output graphics information.

Are there any way to get the unity GUI output normally when run program in headless linux server's docker without adding "-batchmode -nographics" to the unity start command.

Really desire for a response. Many thanks!!

[mikezhang95]

Hi, do you have any ideas about how to do this on headless server?

[Tushar-N]

Is there a workaround for this? I'm also looking for a way to run headless. I still need to process the image, just not render it on a screen since my servers don't have xorg or docker. I tried:

• headless=true for the controller, but no image is created in this case. Same with -nographics for unity.
• Xvfb (or xvfb-run) to create a virtual display, but I get this error, although glxgears and glxinfo runs fine with the virtual display.

  Desktop is 800 x 600 @ 0 Hz
  Unable to find a supported OpenGL core profile
  Failed to create valid graphics context: please ensure you meet the minimum requirements
  E.g. OpenGL core profile 3.2 or later for OpenGL Core renderer
  Vulkan detection: 0
  No supported renderers found, exiting
  (Filename:  Line: 634)

Related issues: https://github.com/allenai/ai2thor/issues/41, https://github.com/allenai/ai2thor/issues/21

[Tushar-N]

Looks like there's some sort of incompatibility with Ubuntu 18.04 and Xvfb-1.19.6. Following the instructions at this issue from the ml-agents repo fixes the issue for me. The following runs now: xvfb-run --auto-servernum --server-args='-screen 0 640x480x24' python run.py

[shuyanzhou]

@li3cmz Hi, did you solve this problem? Even I did not get an error on controller.start(), I notice controller.last_event.frame is empty. Should this be expected when docker_enable=True?

[vmurahari3]

@Tushar-N what fps do you get by using xvfb?

[Tushar-N]

Around 20FPS with multiple instances running (8 envs in parallel). If I render on GPU, I can get ~40FPS.

[vmurahari3]

@Tushar-N Do you get ~40 FPS on a GPU with a single thread or with 8 envs in parallel?

Really appreciate your prompt responses. Thank you!

[ekolve]

Please take a look at this: https://github.com/allenai/ai2thor-docker

It is a mini-framework to make running ai2thor in docker easier.

[mattdeitke]

Closing as this appears to be solved with AI2-THOR Docker. Feel free to re-open if there are any related follow-ups :)