[Question for the runtime error] OpenXR-Loader failed to determine active runtime

[PARKBONG]

Hi, I've encountered some problems while running your example code.

1. Warning - It occurs on the xr.init() 1.1. [WARNING] omni.kit.viewport.get_viewport_interface: module 'omni.kit.viewport' has no attribute 'get_viewport_interface' 1.2. [WARNING] Using legacy omni.kit.viewport_legacy.get_viewport_interface

Q1. In the case of 1.1, I think it is related with migration issue maybe, Will that cause a problem? I tried an example in IsaacSim 2022.1.1 version.

2. Error - It occurs on the xr.create_instance() 2.1. Error [GENERAL | | OpenXR-Loader] : RuntimeManifestFile::FindManifestFiles - failed to determine active runtime file path for this environment

Q2. I tried to perform the example with SteamVR like the example video you provided. But I have no idea why the openxr cannot detect steam runtime. How can I solve the runtime error? I'm attaching my screenshot and CLI log (left side of my screenshot).

CLI_log.txt

• Q3. As a beginner, I would like to ask where can I check the entire log.

Thank you!

[PARKBONG]

And I add a few information. System ---------------------------- OS : Ubuntu 20.04 LTS IsaacSim version : 2022.1.1 HMD : Vive Pro 2 Driver : https://github.com/CertainLach/VivePro2-Linux-Driver / proxy driver that from oculus

Is there a possibility that the runtime error is related to this? I think the error is not hardware related because SteamVR still works... If your code is only for the Vive pro 1, please tell me. thanks!

[Toni-SM]

Hi @PARKBONG

I brought the HTV Vivie with me to do some testing for this weekend... Regarding the comments/issues:

1. Legacy viewport warning

   Those warnings can be ignored. In any case, I will change this warning for info to indicate when the legacy API is in use.

2. RuntimeManifestFile::FindManifestFiles - failed to determine active runtime file path for this environment

   Please, make sure there is a default OpenXR runtime configured, as indicated here: https://github.com/KhronosGroup/OpenXR-SDK-Source/issues/222#issuecomment-728431926

   A way to check/set it could be using the StreamVR settings...

   

3. Full log

   Because the Omniverse C/C++ API is not released yet, I am not logging using carb... Currently, this extension shows all info by printing in the terminal. To get more detailed output, set the following environment variables in the terminal prior to running Isaac Sim

   export XR_LOADER_DEBUG=all
   export VR_LOG_DEBUG=1

4. Supported devices

   I have only tested the extension with the HTV Vive Pro, it's what I have at hand... However, the extension is just a compact binging for OpenXR standard. Then all devices that support that standard can be handled from the extension (in principle 😅)

Hope this first round be useful. Feel free to continue the thread or open a new issue if problems arise/continue :)

[PARKBONG]

To @Toni-SM

I appreciate your quick and friendly feedback :-) In my case, just setting the current OpenXR runtime as SteamVR solved my problem as your second answer.

But after running the example code, I faced another problem.

[Error] VkResult: ERROR_OUT_OF_DEVICE_MEMORY

which is caused when I run xr.setup_streo_view().. I concluded that this error is caused by stereo image generation of Viewpoint, which exceeded the memory of my hardware specification (i.e., RTX 3070Ti, with 8GB VRAM).

Here, with two viewpoints, memory error occurs.

After the error, I closed one Viewpoint manually, there is no memory error, but HMD doesn't work.

CLI log. CLI_log.txt

Q1. Can you share your graphic card specification? I wonder how much is the minimum required specification.

Q2. Is there a way to run the example code without out-of-memory issue in my hardware specification? For example, can I configure the system to have two images (i.e., stereo image) internally, by setting Viewpoint as headless? I'm not sure if it solves the memory issue, but it may be worth trying.

Thank you for your support!

[Toni-SM]

Hi @PARKBONG

1. graphic card specification

    |---------------------------------------------------------------------------------------------|
    | Driver Version: 515.65.1      | Graphics API: Vulkan
    |=============================================================================================|
    | GPU | Name                             | Active | LDA | GPU Memory | Vendor-ID | LUID       |
    |     |                                  |        |     |            | Device-ID | UUID       |
    |---------------------------------------------------------------------------------------------|
    | 0   | NVIDIA GeForce RTX 3080 Laptop.. | Yes: 0 |     | 16384   MB | 10de      | 0          |
    |     |                                  |        |     |            | 249c      | a09ab121.. |
    |---------------------------------------------------------------------------------------------|
    | 1   | Intel(R) UHD Graphics (TGL GT1)  |        |     | 23835   MB | 8086      | 0          |
    |     |                                  |        |     |            | 9a60      | ee98bced.. |
    |=============================================================================================|
    | OS: Linux HP-ZBook-Studio-G8, Version: 5.10.0-1044-oem
    | XServer Vendor: The X.Org Foundation, XServer Version: 12013000 (1.20.13.0)
    | Processor: 11th Gen Intel(R) Core(TM) i9-11950H @ 2.60GHz | Cores: Unknown | Logical: 16
    |---------------------------------------------------------------------------------------------|
    | Total Memory (MB): 31780 | Free Memory: 24798
    | Total Page/Swap (MB): 32767 | Free Page/Swap: 32767
    |---------------------------------------------------------------------------------------------|

2. ERROR_OUT_OF_DEVICE_MEMORY

   For getting the images to render to the HMD it is necessary to have the viewport windows opened and this consumes a lot of memory. The viewport windows are created with the same HMD resolution (in your case 2468 x 2740, which is higher than the default viewport resolution in Omniverse). An alternative for consuming less memory is to reduce the HDM resolution in StreamVR Settings...

   

   If you want to run without rendering to the HMD, please follow this post: https://forums.developer.nvidia.com/t/external-extensions-openxr-compact-binding-for-creating-extended-reality-applications/192173/15?u=toni.sm

3. Extension update

   I have modified the extension code to initialize Omniverse sensors before calling them for rendering. This fixes a visualization error in the latest Omniverse version. If you want to test it, because it is not released yet, download this file: https://github.com/Toni-SM/semu.xr.openxr/blob/main/src/semu.xr.openxr/semu/xr/openxr/openxr.py and replace the semu.xr.openxr/semu/xr/openxr/_openxr.cpython-37m-x86_64-linux-gnu.so file with it

[PARKBONG]

For me, changing resolution solved the issue!

I appreciate your support.

Actually, I have another question, Can you tell me what should I do when I stuck in the Next Up window?

[PARKBONG]

Press the play button!

[Toni-SM]

Hi @PARKBONG

Nice. Yes, in the snippet, the OpenXR loop runs inside the physics event, then, it is necessary to play the simulation for that event to trigger...

By the way, I just published a new version (v0.0.4-beta) in which you can configure the scene unit, necessary to operate with different NVIDIA applications, for example, Isaac Sim (in meters by default) or Code (in centimeters by default). In addition, I have updated the snippet that appears in the README and the example video