smlpt
commented
1 year ago Managed to kind of fix this by rebinding the controls using the OVR Advanced Settings addon and following this guide to emulate Oculus Touch controllers. I also had to change the settings to render the controllers models as Oculus Touch controllers for collision detection to work properly. Now the demo scene runs fine.
This was a somewhat tedious process and I don't understand why it didn't work out of the box. The default bindings tried to emulate a trackpad using the B button, which resulted in the behavior described in the issue above. I think for usability reasons, a complete manual rebinding is not feasible for end users.
And the issue that the actual controller models don't have collision detection is also something we should try to fix.
This bug does not affect all buttons, but it seems that it affects the joysticks. Also, issue seems to be related only to the Reverb; headsets like the Vive work fine.
Using the VRControllerExample, the following behavior can be observed:
In the VolumeCroppingExample, the behavior looks as follows:
By logging the OpenVRButton event in the OpenVRHMD class, I get the following results: (some buttons trigger two events)
The debug log tells me that joystick inputs are indeed registered as a ButtonTouch event, but the log does not output any specific button name for it.
Attached is the log when running the VRControllerExample.
Log
``` [main] INFO OpenVRHMD - Initializing OpenVR (1)... [main] INFO OpenVRHMD - OpenVR library: Initialized. [main] INFO OpenVRHMD - Compositor initialized [main] INFO OpenVRHMD - Initialized device WindowsMR holographic with render target size 2324x2272 [main] INFO VulkanRenderer - Loaded Deferred Shading (Deferred Shading, with HDR postprocessing and FXAA) [main] INFO VulkanRenderer - Creating Vulkan instance with extensions VK_KHR_win32_surface,VK_KHR_surface,VK_KHR_surface,VK_KHR_win32_surface,VK_KHR_external_memory_capabilities,VK_KHR_get_physical_device_properties2,VK_KHR_surface,VK_KHR_win32_surface,VK_NV_external_memory_capabilities and layers [main] INFO VulkanRenderer - Available special-purpose swapchains are: SwingSwapchain, HeadlessSwapchain, OpenGLSwapchain [main] INFO Companion - Physical devices: [main] INFO Companion - 0: Nvidia NVIDIA GeForce RTX 2070 with Max-Q Design (DiscreteGPU, driver version -496.196.0, Vulkan API 1.3.224) (selected) [main] INFO Companion - 1: Intel Intel(R) UHD Graphics 630 (IntegratedGPU, driver version 0.404.1660, Vulkan API 1.3.205) [main] INFO VulkanDevice - Creating 2 distinct queue groups [main] INFO VulkanRenderer - Using default swapchain [main] INFO VulkanRenderer - Renderer initialisation complete. [main] INFO VulkanRenderer - Toggling VR! [main] INFO VulkanRenderer - Loaded Deferred Shading, Stereo (OpenGL/Vulkan shader pipeline implementing deferred shading, with stereo rendering) [main] INFO PointLight - Resetting light radius [main] INFO PointLight - Resetting light radius [main] INFO PointLight - Resetting light radius [main] INFO PointLight - Resetting light radius [main] INFO InputHandler - No custom key configuration found, using default keybindings. [main] INFO VRControllerExample - Adding behaviour move_back bound to S to HMD [main] INFO VRControllerExample - Adding behaviour move_right bound to D to HMD [main] INFO VRControllerExample - Adding behaviour move_forward bound to W to HMD [main] INFO VRControllerExample - Adding behaviour move_left bound to A to HMD [main] INFO VulkanRenderer - Recreating Swapchain at frame 0 (VulkanSwapchain) [main] INFO VulkanSwapchain - Present queue is 0, graphics queue is 0 [main] INFO VulkanSwapchain - Selected present mode: VK_PRESENT_MODE_FIFO_KHR with 3 images [main] INFO Companion - Creating render framebuffer GeometryBuffer for pass rightEye (3844x1055) [main] INFO Companion - + attachment NormalsMaterial, RGBA_Float16 [main] INFO Companion - + attachment DiffuseAlbedo, RGBA_UInt8 [main] INFO Companion - + attachment ZBuffer, Depth32 [main] INFO Companion - Creating render framebuffer GeometryBuffer for pass leftEye (3844x1055) [main] INFO Companion - Reusing already created framebuffer [main] INFO Companion - Creating render framebuffer AOTemp1 for pass AO (3844x1055) [main] INFO Companion - + attachment Occlusion, R_UInt8 [main] INFO Companion - Creating render framebuffer AOTemp2 for pass AOBlurV (3844x1055) [main] INFO Companion - + attachment Occlusion, R_UInt8 [main] INFO Companion - Creating render framebuffer AOBuffer for pass AOBlurH (1922x527) [main] INFO Companion - + attachment Occlusion, R_UInt8 [main] INFO Companion - Creating render framebuffer ForwardBuffer for pass DeferredLightingRight (3844x1055) [main] INFO Companion - + attachment Color, RGBA_Float16 [main] INFO Companion - Creating render framebuffer ForwardBuffer for pass DeferredLightingLeft (3844x1055) [main] INFO Companion - Reusing already created framebuffer [main] INFO Companion - Creating render framebuffer HDRBuffer for pass ForwardShadingRight (3844x1055) [main] INFO Companion - + attachment Color, RGBA_Float16 [main] INFO Companion - + attachment Depth, Depth32 [main] INFO Companion - Creating render framebuffer HDRBuffer for pass ForwardShadingLeft (3844x1055) [main] INFO Companion - Reusing already created framebuffer [main] INFO Companion - Creating render framebuffer FXAABuffer for pass HDR (3844x1055) [main] INFO Companion - + attachment Color, RGBA_UInt8 [main] INFO VulkanRenderer - Scene initialization started. [main] INFO VulkanRenderer - Scene initialization complete, took 83.01906 ms. [DefaultDispatcher-worker-2] INFO OpenVRHMD - Loading model for generic_hmd from E:\Steam\steamapps\common\SteamVR\resources\rendermodels\generic_hmd\generic_hmd.obj [DefaultDispatcher-worker-1] INFO OpenVRHMD - Loading model for C:\Users\Samuel\AppData\Local\Microsoft/Windows/OpenVR\controller_1642_1118_1\controller.obj from C:\Users\Samuel\AppData\Local\Microsoft/Windows/OpenVR\controller_1642_1118_1\controller.obj [DefaultDispatcher-worker-1] INFO OpenVRHMD - Loading model from composite JSON, C:\Users\Samuel\AppData\Local\Microsoft\Windows\OpenVR\controller_1642_1118_1\controller.json [DefaultDispatcher-worker-2] INFO Mesh - Importing geometry from OBJ file E:/Steam/steamapps/common/SteamVR/resources/rendermodels/generic_hmd/generic_hmd.obj [DefaultDispatcher-worker-2] INFO Mesh - Starting preparse [DefaultDispatcher-worker-3] INFO OpenVRHMD - Loading model for C:\Users\Samuel\AppData\Local\Microsoft/Windows/OpenVR\controller_1642_1118_2\controller.obj from C:\Users\Samuel\AppData\Local\Microsoft/Windows/OpenVR\controller_1642_1118_2\controller.obj [DefaultDispatcher-worker-3] INFO OpenVRHMD - Loading model from composite JSON, C:\Users\Samuel\AppData\Local\Microsoft\Windows\OpenVR\controller_1642_1118_2\controller.json [DefaultDispatcher-worker-2] INFO Mesh - Preparse took 70.3565 ms [DefaultDispatcher-worker-1] ERROR OpenVRHMD - Exception: com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class graphics.scenery.controls.OpenVRHMD$CompositeModel] value failed for JSON property thumbnail due to missing (therefore NULL) value for creator parameter thumbnail which is a non-nullable type at [Source: (StringReader); line: 275, column: 1] (through reference chain: graphics.scenery.controls.OpenVRHMD$CompositeModel["thumbnail"]) [DefaultDispatcher-worker-1] INFO OpenVRHMD - Loading composite JSON failed, trying to fall back to regular model. [DefaultDispatcher-worker-3] ERROR OpenVRHMD - Exception: com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class graphics.scenery.controls.OpenVRHMD$CompositeModel] value failed for JSON property thumbnail due to missing (therefore NULL) value for creator parameter thumbnail which is a non-nullable type at [Source: (StringReader); line: 275, column: 1] (through reference chain: graphics.scenery.controls.OpenVRHMD$CompositeModel["thumbnail"]) [DefaultDispatcher-worker-3] INFO OpenVRHMD - Loading composite JSON failed, trying to fall back to regular model. [DefaultDispatcher-worker-3] INFO Mesh - Importing geometry from OBJ file C:/Users/Samuel/AppData/Local/Microsoft/Windows/OpenVR/controller_1642_1118_2/controller.obj [DefaultDispatcher-worker-3] INFO Mesh - Starting preparse [DefaultDispatcher-worker-1] INFO Mesh - Importing geometry from OBJ file C:/Users/Samuel/AppData/Local/Microsoft/Windows/OpenVR/controller_1642_1118_1/controller.obj [DefaultDispatcher-worker-1] INFO Mesh - Starting preparse [DefaultDispatcher-worker-3] INFO Mesh - Preparse took 30.4981 ms [DefaultDispatcher-worker-3] INFO Mesh - Importing materials from MTL file C:/Users/Samuel/AppData/Local/Microsoft/Windows/OpenVR/controller_1642_1118_2/controller.mtl [DefaultDispatcher-worker-1] INFO Mesh - Preparse took 57.1503 ms [DefaultDispatcher-worker-1] INFO Mesh - Importing materials from MTL file C:/Users/Samuel/AppData/Local/Microsoft/Windows/OpenVR/controller_1642_1118_1/controller.mtl [Thread-0] INFO VRControllerExample - Full startup took 29639ms [DefaultDispatcher-worker-2] INFO Mesh - Importing materials from MTL file E:/Steam/steamapps/common/SteamVR/resources/rendermodels/generic_hmd/generic_hmd.mtl [DefaultDispatcher-worker-3] INFO Mesh - Read 4784/4784/4784/20148 v/n/uv/i of model in 1427.5702 ms com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class graphics.scenery.controls.OpenVRHMD$CompositeModel] value failed for JSON property thumbnail due to missing (therefore NULL) value for creator parameter thumbnail which is a non-nullable type at [Source: (StringReader); line: 275, column: 1] (through reference chain: graphics.scenery.controls.OpenVRHMD$CompositeModel["thumbnail"]) at com.fasterxml.jackson.module.kotlin.KotlinValueInstantiator.createFromObjectWith(KotlinValueInstantiator.kt:84) at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:519) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4730) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3677) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3645) at graphics.scenery.controls.OpenVRHMD.loadMeshFromModelPath(OpenVRHMD.kt:934) at graphics.scenery.controls.OpenVRHMD.loadModelForMesh(OpenVRHMD.kt:999) at graphics.scenery.controls.OpenVRHMD$update$d$1$1.invokeSuspend(OpenVRHMD.kt:482) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) [DefaultDispatcher-worker-3] INFO VRControllerExample - Got device Controller-2 at 908510749819300 [DefaultDispatcher-worker-3] INFO OpenVRHMD - Adding child controller.obj(Mesh) to DetachedHeadCamera-graphics.scenery.controls.OpenVRHMD@5875de6a(DetachedHeadCamera) [DefaultDispatcher-worker-2] INFO Mesh - Read 17454/17454/17454/17454 v/n/uv/i of model generic_hmd_generic_hmd_mesh in 1561.3308 ms [DefaultDispatcher-worker-1] INFO Mesh - Read 4784/4784/4784/20148 v/n/uv/i of model in 1468.7984 ms com.fasterxml.jackson.module.kotlin.MissingKotlinParameterException: Instantiation of [simple type, class graphics.scenery.controls.OpenVRHMD$CompositeModel] value failed for JSON property thumbnail due to missing (therefore NULL) value for creator parameter thumbnail which is a non-nullable type at [Source: (StringReader); line: 275, column: 1] (through reference chain: graphics.scenery.controls.OpenVRHMD$CompositeModel["thumbnail"]) at com.fasterxml.jackson.module.kotlin.KotlinValueInstantiator.createFromObjectWith(KotlinValueInstantiator.kt:84) at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:202) at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:519) at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:352) at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185) at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323) at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4730) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3677) at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3645) at graphics.scenery.controls.OpenVRHMD.loadMeshFromModelPath(OpenVRHMD.kt:934) at graphics.scenery.controls.OpenVRHMD.loadModelForMesh(OpenVRHMD.kt:999) at graphics.scenery.controls.OpenVRHMD$update$d$1$1.invokeSuspend(OpenVRHMD.kt:482) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) [DefaultDispatcher-worker-1] INFO VRControllerExample - Got device Controller-1 at 908510691714800 [DefaultDispatcher-worker-1] INFO OpenVRHMD - Adding child controller.obj(Mesh) to DetachedHeadCamera-graphics.scenery.controls.OpenVRHMD@5875de6a(DetachedHeadCamera) Exception in thread "DefaultDispatcher-worker-3" java.util.NoSuchElementException: List is empty. at kotlin.collections.CollectionsKt___CollectionsKt.first(_Collections.kt:214) at graphics.scenery.controls.behaviours.VRTouch$Companion$createAndSet$1.invoke(VRTouch.kt:100) at graphics.scenery.controls.behaviours.VRTouch$Companion$createAndSet$1.invoke(VRTouch.kt:93) at graphics.scenery.controls.OpenVRHMD$update$d$1$1.invokeSuspend(OpenVRHMD.kt:489) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@64f3dfa1, Dispatchers.Default] Exception in thread "DefaultDispatcher-worker-1" java.util.NoSuchElementException: List is empty. at kotlin.collections.CollectionsKt___CollectionsKt.first(_Collections.kt:214) at graphics.scenery.controls.behaviours.VRTouch$Companion$createAndSet$1.invoke(VRTouch.kt:100) at graphics.scenery.controls.behaviours.VRTouch$Companion$createAndSet$1.invoke(VRTouch.kt:93) at graphics.scenery.controls.OpenVRHMD$update$d$1$1.invokeSuspend(OpenVRHMD.kt:489) at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33) at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106) at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:570) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:750) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:677) at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:664) Suppressed: kotlinx.coroutines.DiagnosticCoroutineContextException: [StandaloneCoroutine{Cancelling}@55841527, Dispatchers.Default] ```