search

GodotVR / godot_openvr

GDNative based Open VR module
MIT License
227 stars 35 forks source link

Launching Godot (with ARVR support) Causes Eventual `godot_arvr_process` Seg Fault #56

Open georgewsinger opened 5 years ago

georgewsinger commented 5 years ago

I'm running Godot w/ARVR support on Ubuntu (19.04) + HTC Vive (generation 1). The crash pattern is:

  1. Launch Godot w/o controller(s)

  2. Turn on controller, generating the following console output (but Godot will keep running):

    Found openvr device 1 (vr_controller_vive_1_5)
Creating render model resource
OpenVR: increased use count to 2
Argument count: 1
Searching for: vr_controller_vive_1_5

  3. Click controller, causing the following crash:

    handle_crash: Program crashed with signal 11
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] /lib/x86_64-linux-gnu/libc.so.6(+0x43f60) [0x7f89de6b1f60] (??:0)
[2] /home/george/Simula/addons/godot-openvr/bin/x11/libgodot_openvr.so(+0x2b98) [0x7f89d4abfb98] (??:0)
[3] ARVRInterfaceGDNative::process() (/home/george/SimulaMax/build/godot/modules/gdnative/arvr/arvr_interface_gdnative.cpp:212)
[4] ARVRServer::_process() (/home/george/SimulaMax/build/godot/servers/arvr_server.cpp:352 (discriminator 2))
[5] VisualServerViewport::draw_viewports() (/home/george/SimulaMax/build/godot/servers/visual/visual_server_viewport.cpp:252)
[6] VisualServerRaster::draw(bool, double) (/home/george/SimulaMax/build/godot/servers/visual/visual_server_raster.cpp:107 (discriminator 2))
[7] VisualServerWrapMT::draw(bool, double) (/home/george/SimulaMax/build/godot/servers/visual/visual_server_wrap_mt.cpp:104)
[8] Main::iteration() (/home/george/SimulaMax/build/godot/main/main.cpp:1878)
[9] OS_X11::run() (/home/george/SimulaMax/build/godot/platform/x11/os_x11.cpp:2959)
[10] /home/george/Simula/addons/godot-haskell-plugin/../..//bin/godot(main+0xdc) [0x115b82e] (/home/george/SimulaMax/build/godot/platform/x11/godot_x11.cpp:56)
[11] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f89de694b6b] (??:0)
[12] /home/george/Simula/addons/godot-haskell-plugin/../..//bin/godot(_start+0x2a) [0x115b69a] (??:?)
-- END OF BACKTRACE --

  4. The fuller gdb backtrace output is as follows:

    Found openvr device 1 (vr_controller_vive_1_5)
Creating render model resource
OpenVR: increased use count to 2
Argument count: 1
Searching for: vr_controller_vive_1_5
[New Thread 0x7fffbe195700 (LWP 5188)]

Thread 1 "godot" received signal SIGSEGV, Segmentation fault.
0x00007fffed32cb98 in godot_arvr_process (p_data=0x61074c0) at src/ARVRInterface.cpp:347
347    src/ARVRInterface.cpp: No such file or directory.
(gdb) bt
#0  0x00007fffed32cb98 in godot_arvr_process(void*) (p_data=0x61074c0)
    at src/ARVRInterface.cpp:347
#1  0x000000000154e494 in ARVRInterfaceGDNative::process() (this=0x6112190)
    at modules/gdnative/arvr/arvr_interface_gdnative.cpp:211
#2  0x0000000002df1ac1 in ARVRServer::_process() (this=0x5af5cd0)
    at servers/arvr_server.cpp:356
#3  0x0000000002fea1a0 in VisualServerViewport::draw_viewports() (this=0x4f207b0)
    at servers/visual/visual_server_viewport.cpp:250
#4  0x0000000002fbc351 in VisualServerRaster::draw(bool, double)
    (this=0x4f1db60, p_swap_buffers=true, frame_step=0.019463000819087029)
    at servers/visual/visual_server_raster.cpp:106
#5  0x0000000002ff0bee in VisualServerWrapMT::draw(bool, double)

    at servers/visual/visual_server_wrap_mt.cpp:102
#6  0x000000000119713c in Main::iteration() () at main/main.cpp:1877
#7  0x000000000116a5c9 in OS_X11::run() (this=0x7fffffffdb20)
    at platform/x11/os_x11.cpp:2959
#8  0x000000000115b82e in main(int, char**) (argc=3, argv=0x7fffffffe298)
    at platform/x11/godot_x11.cpp:55

Someone else on a project I'm working on (@KaneTW) inspected libgodot_openvr with debug symbols and found that

arvr_api->godot_arvr_set_controller_button(arvr_data->trackers[event.trackedDeviceIndex], button, true);

is what is crashing, so that this could be a bug with libgodot_openvr.

Does anyone have any thoughts about how to fix this crash?

BastiaanOlij commented 5 years ago

Hmmm, that is weird because if no controllers are attached it shouldn't be calling that. I wonder if this might be related to a recent change in Godot where the default device ID for an ARVRController node is now 1.

I'll have to do some testing to see if I can reproduce this

georgewsinger commented 5 years ago

@BastiaanOlij It turns out that Godot being launched w/o a controller already connected is an incidental data point.

Launching Godot w/Vive controller already connected. This yields, at bootup and with occasional frequency:

ERROR: find_by_type_and_id: Condition ' p_tracker_id == 0 ' is true. returned: __null
   At: servers/arvr_server.cpp:305.

as well as:

Found openvr device 1 (lh_basestation_vive)
Found openvr device 2 (vr_controller_vive_1_5)
Initialized VR interface.
Creating render model resource
OpenVR: increased use count to 2
Argument count: 1
Searching for: vr_controller_vive_1_5

along with the eventual crash reported above (not necessarily from pressing a Vive button either; sometimes pressing a button causes an immediate crash, and sometimes a crash eventually happens regardless). I hope this helps.

georgewsinger commented 5 years ago

Launching Godot w/no controller connected. To make things more complicated (perhaps less complicated?), another person using godot_openvr in our project has reported that launching Godot w/o a controller connected causes (not always, but at least one time):

OpenVR: initialising OpenVR context
Main OpenVR interface has been initialized
Main render models interface has been initialized.
/home/sonata/.steam/steam/steamapps/common/SteamVR/bin/linux64/vrmonitor: error while loading shared libraries: libudev.so.0: cannot open shared object file: No such file or directory
Found openvr device 1 (lh_basestation_vive)
Using breakpad crash handler
Setting breakpad minidump AppID = 250820
Forcing breakpad minidump interfaces to load
Initialized VR interface.
Looking up breakpad interfaces from steamclient
Calling BreakpadMiniDumpSystemInit
ERROR: find_by_type_and_id: Condition ' p_tracker_id == 0 ' is true. returned: __null
   At: servers/arvr_server.cpp:305.
handle_crash: Program crashed with signal 11
Dumping the backtrace. Please include this when reporting the bug on https://github.com/godotengine/godot/issues
[1] /lib/x86_64-linux-gnu/libc.so.6(+0x43f60) [0x7f7adbe9ef60] (??:0)
[2] /home/sonata/workspace/simula/Simula/addons/godot-openvr/bin/x11/libgodot_openvr.so(+0x2b98) [0x7f7ab2f11b98] (??:0)
[3] ARVRInterfaceGDNative::process() (/home/sonata/workspace/simula/Simula/build/godot/modules/gdnative/arvr/arvr_interface_gdnative.cpp:212)
[4] ARVRServer::_process() (/home/sonata/workspace/simula/Simula/build/godot/servers/arvr_server.cpp:352 (discriminator 2))
[5] VisualServerViewport::draw_viewports() (/home/sonata/workspace/simula/Simula/build/godot/servers/visual/visual_server_viewport.cpp:252)
[6] VisualServerRaster::draw(bool, double) (/home/sonata/workspace/simula/Simula/build/godot/servers/visual/visual_server_raster.cpp:107 (discriminator 2))
[7] VisualServerWrapMT::draw(bool, double) (/home/sonata/workspace/simula/Simula/build/godot/servers/visual/visual_server_wrap_mt.cpp:104)
[8] Main::iteration() (/home/sonata/workspace/simula/Simula/build/godot/main/main.cpp:1878)
[9] OS_X11::run() (/home/sonata/workspace/simula/Simula/build/godot/platform/x11/os_x11.cpp:2959)
[10] /home/sonata/workspace/simula/Simula/addons/godot-haskell-plugin/../..//bin/godot(main+0xdc) [0x115c81e] (/home/sonata/workspace/simula/Simula/build/godot/platform/x11/godot_x11.cpp:56)
[11] /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f7adbe81b6b] (??:0)
[12] /home/sonata/workspace/simula/Simula/addons/godot-haskell-plugin/../..//bin/godot(_start+0x2a) [0x115c68a] (??:?)
-- END OF BACKTRACE --

which seems to be fixed by a reboot. Notice that

ERROR: find_by_type_and_id: Condition ' p_tracker_id == 0 ' is true. returned: __null
   At: servers/arvr_server.cpp:305.

coming up again (though to be clear: the p_tracker_id == 0 error doesn't immediately cause the crash; it just floats in the console until the crash eventually happens, either from a button press or not).

BastiaanOlij commented 5 years ago

The p_tracker_id == 0 warning is just a warning, it means there is an ARVRController node somewhere where controller id is set to 0.

I really don't get this crash issue, on my MR headset my controllers go to sleep all the time so I often don't turn them on until the game is running. But I'm on Windows, might be some weird issue in Linux though. @beniwtv do you have any ideas?

Are you running a master build or the version thats in the asset library?

beniwtv commented 5 years ago

I also run often without controllers, and turn them on later. Didn't have the issue so far. I would be good to know what Godot version you are using as well as what version of the asset (library or self-compiled)

georgewsinger commented 5 years ago

Godot version. We are using a fork of Godot based on 10e9221 (commit date: Dec. 22nd 2018). I believe it's a beta version of Godot 3.1.

libgodot_openvr.so + libopenvr_api.so. It turns out we were using fairly old pre-compiled binaries (\~9 months old). I just recompiled godot_openvr and tried to reproduce some of these crashes but couldn't, though I was still getting familiar console output:

ERROR: find_by_type_and_id: Condition ' p_tracker_id == 0 ' is true. returned: __null
   At: servers/arvr_server.cpp:305.
Found openvr device 1 (vr_controller_vive_1_5)
Creating render model resource
OpenVR: increased use count to 2
Argument count: 1
Searching for: vr_controller_vive_1_5
ERROR: find_by_type_and_id: Condition ' p_tracker_id == 0 ' is true. returned: __null
   At: servers/arvr_server.cpp:305.
ERROR: find_by_type_and_id: Condition ' p_tracker_id == 0 ' is true. returned: __null

I'm crossing my fingers and hoping that updating godot_openvr fixed this issue. Right now I'm waiting for another person to test this on their Ubuntu machine. I'll report back with the results.

BastiaanOlij commented 5 years ago

Like i said, the p_tracker_id is just a warning that there is an ARVRController set to an ID for which no controller is active. 0 is the default id (or was) so this is most likely caused by the split second where the ARVRController is added for the new controller but its tracker id isn't set yet.

georgewsinger commented 5 years ago

Unfortunately, someone else testing another Ubuntu machine (with updated godot_openvr libs) is still getting the crashes reported above. :S More specificially, if the godot_openvr crash patterns thus far are:

  1. Launch w/o controller; start controller; click => crash.
  2. Launch w/o controller and it immediately crashes.
  3. Launch w/controller and godot eventually crashes after a few seconds (not necessarily because of a button press).

then the tester said that they're still experiencing (1) and (3), even with updated godot_openvr libs.

beniwtv commented 5 years ago

Could you try the current 3.1 + the current asset and see if the demo works? There have been some breaking changes in GDNative around Godot 3.1 beta/RCs if I remember correctly.

georgewsinger commented 5 years ago

This bug only seems to reproduce when using AMD cards (AMDGPU driver), but hasn't reproduced for nvidia cards running nvidia drivers.

beniwtv commented 5 years ago

I've recently set up am Ubuntu 18.04 container for compiling this on Linux. Godot compiled on newer distros could be the cause of the crash on Ubuntu, which have lower versions of libc.

If you still have this issue, let me know if my recent build in this PR works: https://github.com/GodotVR/godot_openvr/pull/64

georgewsinger commented 5 years ago

@beniwtv I'm using Ubuntu 19.04 and will test out your PR.

  1. What branch of godotengine/godot should I use: master or 3.1`?
  2. Should I use a particular version of libc?
beniwtv commented 5 years ago

1) 3.1 as well as master should be fine. 2) libc is the one from Ubuntu 18.04, so in theory anything equal or newer should work

georgewsinger commented 5 years ago

@beniwtv When attempting to the launch the ./demo in your PR (with godot 3.1 branch), I get the following error:

OpenGL ES 3.0 Renderer: GeForce GTX 1070/PCIe/SSE2
Construct gdnative interface
ERROR: _load_data: Condition ' !f ' is true. returned: ERR_CANT_OPEN
   At: scene/resources/texture.cpp:495.
ERROR: _load: Failed loading resource: res://.import/Milkyway.png-c8038416f38c889c516f7e41b5681a1a.stex
   At: core/io/resource_loader.cpp:285.
ERROR: _load: Failed loading resource: res://Milkyway.png
   At: core/io/resource_loader.cpp:285.
ERROR: poll: res://Main.tscn:4 - Parse Error: [ext_resource] referenced nonexistent resource at: res://Milkyway.png
   At: scene/resources/resource_format_text.cpp:440.
ERROR: load: Condition ' err != OK ' is true. returned: RES()
   At: core/io/resource_loader.cpp:208.
ERROR: _load: Failed loading resource: res://Main.tscn
   At: core/io/resource_loader.cpp:285.
ERROR: start: Failed loading scene: res://Main.tscn
   At: main/main.cpp:1733.
Destruct gdnative interface
WARNING: cleanup: ObjectDB Instances still exist!
   At: core/object.cpp:2089.
ERROR: clear: Resources Still in use at Exit!
   At: core/resource.cpp:425.
beniwtv commented 5 years ago

Do you have a file Milkyway.png there? It seems to be missing according to the output

georgewsinger commented 5 years ago

@beniwtv The main resource should be there:

george@UbuntuBox:~/godot_openvr_pr/demo$ find . -name "*Milkyway*"
./Milkyway.png.import
./Milkyway.png

though it looks like I'm lacking whatever res://.import/Milkyway.png-c8038416f38c889c516f7e41b5681a1a.stex is.

beniwtv commented 5 years ago

ok, try deleting Milkyway.png.import and see if the editor imports it again

georgewsinger commented 5 years ago

ok, try deleting Milkyway.png.import and see if the editor imports it again

The error persists. Strange.