graphdeco-inria / gaussian-splatting

Original reference implementation of "3D Gaussian Splatting for Real-Time Radiance Field Rendering"
https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/
Other
14.65k stars 1.92k forks source link

Question about keyboard controlling in VR mode #974

Open oUp2Uo opened 2 months ago

oUp2Uo commented 2 months ago

Hi, thanks for the great work. I have tried the new VR mode, and I could see the result on VR headset. It is very smooth and has a high FPS.

But I have a question about the keyboard controlling (Seated mode). It seems that key WSAD controls moving in the starting coordinate system, not the direction rotated by the headset. So it will be wired if I rotate my head to see something nearby. For example, key WS will not move front/behind, and moving in a direction with left/right.

Thank you.

cedric-chedaleux commented 2 months ago

Hello @oUp2Uo

Good point! The FPS camera taking care of WSAD controls is not aware about the VR camera positioning and headset orientation.

A new version of SIBR_Viewer with support for OpenXR controllers, some build fixes and a scene repositioning feature is available at https://gitlab.inria.fr/sibr/sibr_core/-/tree/openxr_controllers?ref_type=heads.

Through your controllers or the buttons in the UI you should be able to move within the scene in respect to your headset's orientation. Have a look at the README OpenXR section (https://gitlab.inria.fr/sibr/sibr_core/-/tree/openxr_controllers?ref_type=heads#support-for-vr-headsets-using-openxr-provided-by-orange) to know the controls.

oUp2Uo commented 1 month ago

Hello @cedric-chedaleux Thank you for the reply. I have tried to edit the code these days, and finally was able to fix this.

In short, first we would need to pass the rotation value of headset to InteractiveCameraHandler(in function render of OpenXRRdrMode). Then we could set the rotation value to FPSCamera(in function update of InteractiveCameraHandler. And FPSCamera would need a extra function like setRotation to get the rotation value.). So FPSCamera would be able to process WSAD key event(function moveUsingWASD) with this rotation value. (In order to do this, we need to delete several const in the interface functions, or we could not change variable of reference object.)