search

nvidia-holoscan / holoscan-sdk

The AI sensor processing SDK for low latency streaming workflows
Apache License 2.0
117 stars 31 forks source link

Error when using exclusive display for Holoviz #26

Closed jpmazellier closed 3 months ago

jpmazellier commented 5 months ago

Dear devs,

I'm facing an issue with Holoviz operator when trying to use an exclusive display. In the operator params, I set "use_exclusive_display" as True and "display_name" corresponding to the output of xrandr command. When running the app, the exclusive_window.cpp retrieves the displays (coherent with xrandr) and pick the exclusive display as stated in the corresponding parameter for holoviz operator, but then I get an error message (see below and in the logs after) and the app crashes :

[error] [context.cpp:56] VkResult -13 - unknown [error] [gxf_wrapper.cpp:57] Exception occurred when starting operator: 'holoviz' - Failed to acquire display from X-Server.

I had the issue both on an Orin DevKit (with the iGPU) and on my workstation (x86, dGPU RTX A4000). Maybe there is a special command to run for Vulkan/X-server ? I did not find anything in the documentation of the SDK nor on the internet about such error. Thanks for your help !

Please find the logs below:

Orin (with MST board): [info] [gxf_executor.cpp:247] Creating context [info] [gxf_executor.cpp:1672] Loading extensions from configs... [info] [codec_registry.hpp:314] Replacing existing codec with name 'std::vector'. [warning] [component.cpp:54] Arg 'onnxpath' not found in spec.params() [info] [gxf_executor.cpp:1842] Activating Graph... [info] [gxf_executor.cpp:1874] Running Graph... 2024-05-31 10:29:40.883 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 0] 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 4] 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 5] 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 6] 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 2] 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 1] 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 3] [info] [gxf_executor.cpp:1876] Waiting for completion... 2024-05-31 10:29:40.884 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 7] [info] [context.cpp:50] _ [info] [context.cpp:50] Vulkan Version: [info] [context.cpp:50]  - available:  1.3.204 [info] [context.cpp:50]  - requesting: 1.2.0 [info] [context.cpp:50] __ [info] [context.cpp:50] Used Instance Layers : [info] [context.cpp:50]  [info] [context.cpp:50] Used Instance Extensions : [info] [context.cpp:50] VK_KHR_surface [info] [context.cpp:50] VK_KHR_display [info] [context.cpp:50] VK_EXT_acquire_xlib_display [info] [context.cpp:50] VK_EXT_direct_mode_display [info] [context.cpp:50] VK_EXT_debug_utils [info] [context.cpp:50] VK_KHR_external_memory_capabilities [info] [context.cpp:50] __ [info] [context.cpp:50] Compatible Devices : [info] [context.cpp:50] 0: NVIDIA Tegra Orin (nvgpu) [info] [context.cpp:50] Physical devices found :  [info] [context.cpp:50] 1 [info] [context.cpp:50] ____ [info] [context.cpp:50] Used Device Extensions : [info] [context.cpp:50] VK_KHR_swapchain [info] [context.cpp:50] VK_KHR_external_memory [info] [context.cpp:50] VK_KHR_external_memory_fd [info] [context.cpp:50] VK_KHR_external_semaphore [info] [context.cpp:50] VK_KHR_external_semaphore_fd [info] [context.cpp:50] VK_KHR_push_descriptor [info] [context.cpp:50] VK_EXT_line_rasterization [info] [context.cpp:50]  [info] [vulkan_app.cpp:843] Using device 0: NVIDIA Tegra Orin (nvgpu) (UUID 6e3974cbecf5b21bb253e942f9ac0a4) [info] [exclusive_window.cpp:125] ____ [info] [exclusive_window.cpp:126] Available displays : [info] [exclusive_window.cpp:129] ADA (DP-0.1) [info] [exclusive_window.cpp:129] LG Electronics LG ULTRAWIDE (DP-0.2) [info] [exclusive_window.cpp:134]  [info] [exclusive_window.cpp:135] Using display "LG Electronics LG ULTRAWIDE (DP-0.2)" [info] [exclusivewindow.cpp:148] X server is running, trying to acquire display [error] [context.cpp:56] VkResult -13 - unknown [error] [gxf_wrapper.cpp:57] Exception occurred when starting operator: 'holoviz' - Failed to acquire display from X-Server._ 2024-05-31 10:29:40.993 WARN  gxf/std/entity_executor.cpp@495: Failed to start entity [holoviz] 2024-05-31 10:29:40.993 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E92 named 'holoviz': GXF_FAILURE 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@693: Stopping all async jobs 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@154: Dispatcher thread has no more jobs to check 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@218: Dispatcher thread has stopped checking jobs 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@678: Waiting to join all async threads 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@701: *** DISPATCHER EXEC TIME : 4.828000 ms

2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 2] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 0] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 1] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 5] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 4] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 3] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@370: Event handler thread exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@702: *** DISPATCHER WAIT TIME : 104.676000 ms

2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@703: *** DISPATCHER COUNT : 981

2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 7] exiting. 2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@704: *** WORKER EXEC TIME : 136.834000 ms 

2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@705: *** WORKER WAIT TIME : 626.860000 ms

2024-05-31 10:29:40.993 INFO  gxf/std/multi_thread_scheduler.cpp@706: *** WORKER COUNT : 41

2024-05-31 10:29:40.994 INFO  gxf/std/multi_thread_scheduler.cpp@687: All async worker threads joined, deactivating all entities

Workstation (x86, A4000):  [info] [gxf_executor.cpp:247] Creating context [info] [gxf_executor.cpp:1672] Loading extensions from configs... [warning] [component.cpp:54] Arg 'onnxpath' not found in spec.params() [info] [gxf_executor.cpp:1842] Activating Graph... [info] [gxf_executor.cpp:1874] Running Graph... 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 1] 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 2] 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 4] 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 0] [info] [gxf_executor.cpp:1876] Waiting for completion... 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 5] 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 3] 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 7] 2024-05-31 10:31:09.662 INFO  gxf/std/multi_thread_scheduler.cpp@299: MultiThreadScheduler started worker thread [pool name: default_pool, thread uid: 6] [info] [context.cpp:50] _ [info] [context.cpp:50] Vulkan Version: [info] [context.cpp:50]  - available:  1.3.204 [info] [context.cpp:50]  - requesting: 1.2.0 [info] [context.cpp:50] __ [info] [context.cpp:50] Used Instance Layers : [info] [context.cpp:50]  [info] [context.cpp:50] Used Instance Extensions : [info] [context.cpp:50] VK_KHR_surface [info] [context.cpp:50] VK_KHR_display [info] [context.cpp:50] VK_EXT_acquire_xlib_display [info] [context.cpp:50] VK_EXT_direct_mode_display [info] [context.cpp:50] VK_EXT_debug_utils [info] [context.cpp:50] VK_KHR_external_memory_capabilities [info] [context.cpp:50] __ [info] [context.cpp:50] Compatible Devices : [info] [context.cpp:50] 0: NVIDIA RTX A4000 [warning] [context.cpp:53] Could NOT locate mandatory extension 'VK_KHR_external_semaphore_fd' [warning] [context.cpp:53] Skipping physical device llvmpipe (LLVM 15.0.7, 256 bits) [info] [context.cpp:50] 1: Intel(R) Graphics (RPL-S) [info] [context.cpp:50] Physical devices found :  [info] [context.cpp:50] 2 [info] [context.cpp:50] ____ [info] [context.cpp:50] Used Device Extensions : [info] [context.cpp:50] VK_KHR_swapchain [info] [context.cpp:50] VK_KHR_external_memory [info] [context.cpp:50] VK_KHR_external_memory_fd [info] [context.cpp:50] VK_KHR_external_semaphore [info] [context.cpp:50] VK_KHR_external_semaphore_fd [info] [context.cpp:50] VK_KHR_push_descriptor [info] [context.cpp:50] VK_EXT_line_rasterization [info] [context.cpp:50]  [info] [vulkan_app.cpp:843] Using device 0: NVIDIA RTX A4000 (UUID 31bcc58e3199a1e2fc3411c9e06a3eb1) [info] [exclusive_window.cpp:125] ____ [info] [exclusive_window.cpp:126] Available displays : [info] [exclusive_window.cpp:129] LG Electronics LG ULTRAWIDE (DP-1) [info] [exclusive_window.cpp:129] HP LA2206 (DP-3) [info] [exclusive_window.cpp:134]  [info] [exclusive_window.cpp:135] Using display "HP LA2206 (DP-3)" [info] [exclusivewindow.cpp:148] X server is running, trying to acquire display **[error] [context.cpp:56] VkResult -13 - unknown [error] [gxfwrapper.cpp:57] Exception occurred when starting operator: 'holoviz' - Failed to acquire display from X-Server.** 2024-05-31 10:31:10.412 WARN  gxf/std/entity_executor.cpp@495: Failed to start entity [holoviz] 2024-05-31 10:31:10.412 WARN  gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E92 named 'holoviz': GXF_FAILURE 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@693: Stopping all async jobs 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@154: Dispatcher thread has no more jobs to check 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@218: Dispatcher thread has stopped checking jobs 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@678: Waiting to join all async threads 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@701: *** DISPATCHER EXEC TIME : 15.612000 ms

2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 5] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 6] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@702: *** DISPATCHER WAIT TIME : 734.078000 ms

2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@703: *** DISPATCHER COUNT : 6891

2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@704: *** WORKER EXEC TIME : 131.483000 ms 

2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@705: *** WORKER WAIT TIME : 3689.930000 ms

2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@706: *** WORKER COUNT : 39

2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 2] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 7] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 3] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@370: Event handler thread exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 1] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@315: Worker Thread [pool name: default_pool, thread uid: 0] exiting. 2024-05-31 10:31:10.412 INFO  gxf/std/multi_thread_scheduler.cpp@687: All async worker threads joined,

jpmazellier commented 5 months ago

Dear devs,

i'm just seeing that I did not gave enough info in previous post. So the holoscan version used was 2.0.0 (both for Orin and Worksation). Orin was running under JP6.0 and Workstation under Ubuntu 22.04.

jpmazellier commented 5 months ago

I also just see you released Holoscan 2.1.0 so I went through the documentation on Holoviz and saw the instruction for the exclusive mode display. I did that on my Workstation (see nvidia-setting window attached) Screenshot from 2024-06-05 09-33-14 Hereafter is the xrandr output from my workstation (after disabling my main display, I'm using two displays).

Screen 0: minimum 8 x 8, current 3440 x 1440, maximum 32767 x 32767 DP-0 disconnected (normal left inverted right x axis y axis) DP-1 connected primary 3440x1440+0+0 (normal left inverted right x axis y axis) 879mm x 366mm 3840x2160 29.97 + 25.00 23.98
3440x1440 49.95* 2560x1080 59.94 50.00
1920x1080 60.00 59.94 50.00 29.97 23.98
1280x720 59.94
1024x768 60.00
800x600 60.32
720x576 50.00
720x480 59.94
640x480 59.94 59.93

Then I gave a try again but I still get an error (even if different form HS2.0). Hereafter is the configuration used in the yaml for setting holoviz display properties (DP-1 being the display set in exclusive mode before). Note, same error is if I enable/disable fullscreen here :

display_name: "DP-1" use_exclusive_display: true fullscreen: true

Please find below the log of the application that crashed (using Holoscan 2.1.0):

[info] [context.cpp:50] Used Instance Layers : [info] [context.cpp:50] [info] [context.cpp:50] Used Instance Extensions : [info] [context.cpp:50] VK_KHR_surface [info] [context.cpp:50] VK_KHR_display [info] [context.cpp:50] VK_EXT_direct_mode_display [info] [context.cpp:50] VK_EXT_debug_utils [info] [context.cpp:50] VK_KHR_external_memory_capabilities [info] [context.cpp:50] ____ [info] [context.cpp:50] Compatible Devices : [info] [context.cpp:50] 0: NVIDIA RTX A4000 [warning] [context.cpp:53] Could NOT locate mandatory extension 'VK_KHR_external_semaphore_fd' [warning] [context.cpp:53] Skipping physical device llvmpipe (LLVM 15.0.7, 256 bits) [info] [context.cpp:50] 1: Intel(R) Graphics (RPL-S) [info] [context.cpp:50] Physical devices found : [info] [context.cpp:50] 2 [info] [context.cpp:50] ____ [info] [context.cpp:50] Used Device Extensions : [info] [context.cpp:50] VK_KHR_swapchain [info] [context.cpp:50] VK_KHR_external_memory [info] [context.cpp:50] VK_KHR_external_memory_fd [info] [context.cpp:50] VK_KHR_external_semaphore [info] [context.cpp:50] VK_KHR_external_semaphore_fd [info] [context.cpp:50] VK_KHR_push_descriptor [info] [context.cpp:50] VK_EXT_line_rasterization [info] [context.cpp:50] [info] [vulkan_app.cpp:845] Using device 0: NVIDIA RTX A4000 (UUID 31bcc58e3199a1e2fc3411c9e06a3eb1) [info] [exclusive_window.cpp:120] ____ [info] [exclusive_window.cpp:121] Available displays : [info] [exclusive_window.cpp:124] LG Electronics LG ULTRAWIDE (DP-1) [info] [exclusive_window.cpp:129] [info] [exclusive_window.cpp:130] Using display "LG Electronics LG ULTRAWIDE (DP-1)" [info] [exclusive_window.cpp:161] Using display mode 1920x1080 60.000 Hz [error] [context.cpp:56] /workspace/holoscan-sdk/modules/holoviz/thirdparty/nvpro_core/nvvk/swapchain_vk.cpp(172): Vulkan Error : unknown [error] [context.cpp:56] /workspace/holoscan-sdk/modules/holoviz/thirdparty/nvpro_core/nvvk/swapchain_vk.cpp(172): Vulkan Error : unknown [error] [context.cpp:56] /workspace/holoscan-sdk/modules/holoviz/thirdparty/nvpro_core/nvvk/swapchain_vk.cpp(172): Vulkan Error : unknown [error] [gxf_wrapper.cpp:57] Exception occurred when starting operator: 'holoviz' - Failed to update swap chain. 2024-06-05 09:28:25.696 WARN gxf/std/entity_executor.cpp@495: Failed to start entity [holoviz] 2024-06-05 09:28:25.696 WARN gxf/std/multi_thread_scheduler.cpp@342: Error while executing entity E65 named 'holoviz': GXF_FAILURE 2024-06-05 09:28:25.696 INFO gxf/std/multi_thread_scheduler.cpp@693: Stopping all async jobs

AndreasHeumann commented 5 months ago

Hi, which GPU driver versions are you using? Basically the steps you mentioned seem to be correct.

I've seen that you have set both use_exclusive_display and fullscreen. Exclusive display is always fullscreen (since there is no desktop manager and no windows) but I don't know if there is a problem when setting both flags. Could you check with fullscreen set to false?

use_exclusive_display: true
fullscreen: true
AndreasHeumann commented 5 months ago

I just tested it, setting both flags is not a problem. Exclusive display still works.

tbirdso commented 4 months ago

Hi @jpmazellier , can you please confirm whether @AndreasHeumann 's suggestion above has resolved your issue?

tbirdso commented 3 months ago

Closing this issue as inactive. Please re-open if you observe the issue after setting the configurations suggested above.