VR not working in Linux

[michalvavrecka]

Hello,

while trying to run iGibson in Ubuntu 20.04 with HTC VivePro (SteamVR, OpenVR runnng) I am facing problems with renderer.

Although Linux system detected, it runs Windows option in mesh_renderer_cpu.py line 106

elif self.platform == "Windows" or self.class.name == "MeshRendererXVR": from igibson.render.mesh_renderer import VRRendererContext # type: ignore

adopting VRRendererContext, while it is not able to localize libopenvr_api.so

ImportError: libopenvr_api.so: cannot open shared object file: No such file or directory

When disabling windows option and run Linux with EGLRendererContext it will miss some functions

AttributeError: 'igibson.render.mesh_renderer.EGLRendererContext.EG' object has no attribute 'preRenderVR'

My question is simple - what Renderer is for Linux machine? Where to put libopenvr_api.so to be visible by VRRendererContext?

Thanks for answer

Michal

[mjlbach]

Did you build with USE_VR=1 pip install -e .

[michalvavrecka]

Yes, the installation was without errors. I do not know, how to debug it, as I am not sure, what renderer is adopted in Linux. In Cmakelist.txt is libopenvr_api.so bind to VRRenderContext

find_library( OPENVR_LIBRARIES NAMES openvr_api PATHS "${OPENVR_DIR}/bin" "${OPENVR_DIR}/lib" PATH_SUFFIXES ${PATH_SUFFIXES} NO_DEFAULT_PATH NO_CMAKE_FIND_ROOT_PATH)

else() target_link_libraries( VRRendererContext PRIVATE pybind11::module ${CMAKE_DL_LIBS} glfw ${GLFW_LIBRARIES} ${OPENGL_LIBRARIES} ${OPENVR_LIBRARIES} ${CUDA_LIBRARIES} cryptopp-static)

but in the documentation is EGL mentioned for Linux VR. There is also USE_EGL in Cmakelist, but there is not any documentation to it. Please let me know, whether libopenvr_api.so is mandatory for Linux.

[mjlbach]

In linux, the VRRendererContext is supposed to be used when compiled with VR support. I'm not sure what happened, but I am guessing a pip/setuptools update changed how files are copied from /tmp during the build process. libopenvr_api.so.used to be automatically fetched by this line https://github.com/StanfordVL/iGibson/blob/3ad2aefabcf1f370ef3dabf704d0ac3becce0df9/igibson/render/CMakeLists.txt#L95-L99

Maybe @wensi-ai who has recently worked with cleaning up our cmake files can suggest how to fix it.

[michalvavrecka]

Thanks for your answer. I tested the VR in Windows and it works (you need to git clone openvr into render folder). Also the Linux version compiles openvr (you need to change render/Cmakelist.txt to point to local openvr folder). Unfortunatelly VRRendererContext still fails to work. I am facing

ImportError: /home/michal/code/iGibson/igibson/render/mesh_renderer/VRRendererContext.cpython-38-x86_64-linux-gnu.so: undefined symbol: cudaMemcpy2DFromArray

I downgraded to 470 drivers, installed all the cuda dependecies mentioned in documentation, but this error still persists. Can you provide me info about the Nvidia and Cuda versions working under Linux? Is docker image VR-Linux compatible?

Thanks for answer

Michal

[mjlbach]

You can just disable USE_CUDA in the cmake file, this is usually an error linking with the particular pytorch/cuda version

[michalvavrecka]

Thanks, I already tried this option, but then new error appeared. There is a problem with VR runtime initialization:

Unable to initialize VR runtime.

This is unfortunatelly part of C++ part of VRRendererContext.cpp , so it is uneasy to debug it.

The erorr appear with/without torch installed, when steamVR is running or not.

When I changed VRRendererContext.cpp to show me the error code 102, that equals according to openvr_api.json to

{"name": "VRInitError_Init_VRClientDLLNotFound","value": "102"}

Does it mean, that it still does not see the libopenvr_api.so even the installation was without errors?

My present local modification of Cmakelist.txt is as follows:

if(USE_VR)

Find OpenVR

include(FetchContent) FetchContent_Declare(openvr GIT_REPOSITORY https://github.com/ValveSoftware/openvr.git GIT_TAG v1.14.15 ) FetchContent_MakeAvailable(openvr)

set(OPENVR_DIR "/home/michal/code/iGibson/igibson/render/openvr/") set(OPENVR_INCLUDE_DIR "${OPENVR_DIR}/headers") include_directories("${OPENVR_INCLUDE_DIR}")