Connecting to ai2thor without ssh x11forwarding

[IgorDroz]

Hey,

My fps is very low when i interact with ai2thor, i guess the reason is that i work remotely and i use ssh x11 forwarding. therfore all the communication between my local computer and the remote vm is the bottleneck. currently i set DISPLAY=localhost:10.0 because DISPLAY=:0.0 doesnt work for me and it shows me this unity gui on my display again...

How can i run ai2thor faster? without opening unity gui on my side

Thanks!

[ekolve]

Are you able to run a X11 server within your VM? I don't understanding what machine you are referring to when you say "my side". Do you mean that when you ssh to your VM and you set the DISPLAY on your VM to 0.0 it still shows back on your desktop that you originated the ssh connection from?

[IgorDroz]

Are you able to run a X11 server within your VM? I don't understanding what machine you are referring to when you say "my side". Do you mean that when you ssh to your VM and you set the DISPLAY on your VM to 0.0 it still shows back on your desktop that you originated the ssh connection from?

Yes, i have x11 server on my vm. And yes, i meant that when i ssh to my vm and set the display to localhost:10.0 or just :10.0 it opens on my desktop a unity window with the gui of ai2thor which i suspect slows down my interaction with ai2thor. It there a way to make the x11 run on a virtual display/background so it doesnt open a unity window on my desktop? or alternatively, is there any other suggestion to speed up the performances? because currently it is very slow.. under 1 fps

[ekolve]

Could you try the following experiment? Can you try setting your DISPLAY to use your vm's X11 server and run glxgears (provided by the mesa-utils package on ubuntu) and see if it is rendering on your local desktop or on your remote VM?

[IgorDroz]

Could you try the following experiment? Can you try setting your DISPLAY to use your vm's X11 server and run glxgears (provided by the mesa-utils package on ubuntu) and see if it is rendering on your local desktop or on your remote VM?

Okay i did it and got the following:

When i set DISPLAY=:10.0 then it is rendering on my desktop with about 18 fps. When i set DISPLAY=:0 then it its not rendering on my desktop but i get much better fps of 570~

I guess DISPLAY=:0 is what i need to use, but when i set it and run the code i get the following error:

File "/home/igor.drozdov/miniconda3/envs/alfred_env/lib/python3.6/site-packages/ai2thor/controller.py", line 697, in _start_unity_thread raise Exception("command: %s exited with %s" % (command, returncode)) Exception: command: ['/home/igor.drozdov/.ai2thor/releases/thor-201909061227-Linux64/thor-201909061227-Linux64', '-screen-fullscreen', '0', '-screen-quality', '4', '-screen-width', '300', '-screen-height', '300'] exited with 1

[IgorDroz]

The output of the Player.log is:

Desktop is 1360 x 768 @ 60 Hz Unable to find a supported OpenGL core profile Failed to create valid graphics context: please ensure you meet the minimum requirements E.g. OpenGL core profile 3.2 or later for OpenGL Core renderer Vulkan detection: 0 No supported renderers found, exiting

(Filename: Line: 634)

[IgorDroz]

This is the output of glxinfo:

$ glxinfo | grep OpenGL OpenGL vendor string: VMware, Inc. OpenGL renderer string: llvmpipe (LLVM 10.0.0, 256 bits) OpenGL core profile version string: 3.3 (Core Profile) Mesa 20.0.8 OpenGL core profile shading language version string: 3.30 OpenGL core profile context flags: (none) OpenGL core profile profile mask: core profile OpenGL core profile extensions: OpenGL version string: 3.1 Mesa 20.0.8 OpenGL shading language version string: 1.40 OpenGL context flags: (none) OpenGL extensions: OpenGL ES profile version string: OpenGL ES 3.1 Mesa 20.0.8 OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.10 OpenGL ES profile extensions:

[IgorDroz]

Hi again,

I solved the issue. But it is still not as fast as i thought, seems like it take 0.5 seconds for each call of step() of the controller. How can i check that i run ai2thor on GPU and not on CPU?

[xyzhu123]

Hi @IgorDroz , could you please elaborate on how did you solve the bug? I recently encountered the same one