Controller model representation

[smrghsh]

Nature of issue?

• [ ] Found a bug
• [ ] Existing feature enhancement
• [x] New feature request

Which area does this problem relate to?

• [x] VR
• [ ] AR(ARCORE)
• [ ] AR(MARKER)

New feature details:

Improved Controller Representation

The following four are crucial XR features where ease of implementation benefits prototyping speed and removes barriers to development. I have listed commonly accessed properties and calculations that must be performed.

1. Navigating menus with raycasting - Controller position and orientation, Primary button access
2. Teleporting - Controller position and orientation, Joystick access, secondary button access
3. Grabbing - Controller grip button or secondary button access, Controller orientation
4. Physics based interactions -Controller speed, controller acceleration

Challenges

Devices have varying implementations and real-life appearances— Therefore, the controller representation would benefit from a single visual virtual representation and wrapper API. In an ideal world, this would mimic the simplicity of p5’s globally scoped mouse position.

p5.xr must account for intermittent device connection — This is a challenging one, as p5.xr should be usable by those without intermediate knowledge of error handling. p5 must return a default position for the controllers— so in the interim, I suggest 0,0,0. More advice needed from accessibility experts

Solution

• [ ] Expand Manual Test for Import to account for button presses by adding primitives and control flow to the testing protocol. Currently:

Goal:

• [ ] Expand Manual Test to Provide visual virtual representation of controller properties and common calculations, likely in metric format to align with the WebXR Device API Spec

• [ ] Write documentation with example and follow up with #158

• [ ] Provide speed and acceleration metrics with manual test

[smrghsh]

@stalgiag Updated with details

[stalgiag]

Great! Thanks for writing this up!

Add more info about the proposed usage once you have a design that you are prepared to move forward with. For example, you mentioned 'the simplicity of p5’s globally scoped mouse position' - what is the equivalent in XR? Also, raycast coords on menus would be most useful as 2D coordinates (much like mouseX and mouseY), how do we provide something simple but useful for this?