Unable to use XR components out of the box

[itsdouges]

Originally raised by @verekia.

The following code causes a useXR must be used within an <XR /> component! error:

import { Interactive } from '@react-three/xr'

const Plane = () => (
  <Interactive onSelect={() => console.log('Hello')}>
    <mesh>
      <meshStandardMaterial color="red" />
      <planeGeometry />
    </mesh>
  </Interactive>
)

XR.tsx:399 Uncaught Error: useXR must be used within an <XR /> component!
    at useXR (XR.tsx:399:21)
    at useInteraction (Interactions.tsx:149:26)
    at Interactive2 (Interactions.tsx:197:18)
    at renderWithHooks (react-reconciler.development.js:7363:18)
    at updateForwardRef (react-reconciler.development.js:11457:20)
    at beginWork (react-reconciler.development.js:13880:16)
    at HTMLUnknownElement.callCallback2 (react-reconciler.development.js:14219:14)
    at Object.invokeGuardedCallbackDev (react-reconciler.development.js:14268:16)
    at invokeGuardedCallback (react-reconciler.development.js:14329:31)
    at beginWork$1 (react-reconciler.development.js:19537:7)

Versions used:

"@react-three/drei": "9.48.1",
"@react-three/fiber": "8.9.1",
"@react-three/xr": "5.1.2",
"@triplex/run": "0.25.0",
"react": "18.2.0",
"three": "0.148.0",

See: https://github.com/pmndrs/react-xr#xr

[itsdouges]

By @verekia:

Relevant lines: https://github.com/pmndrs/react-xr/blob/master/src/XR.tsx#L29 https://github.com/pmndrs/react-xr/blob/master/src/XR.tsx#L431

const XRContext = React.createContext<UseBoundStore<XRState>>(null!)

// ...

export function XR(props: XRProps) {
 // ...
  return (
    <XRContext.Provider value={store}>
      <XRManager {...props} />
    </XRContext.Provider>
  )
}

// ...

export function useXR() {
  // ...
  const store = React.useContext(XRContext)
  if (!store) throw new Error('useXR must be used within an <XR /> component!')

Would it work to export that const XRContext?

[itsdouges]

By @verekia:

My projects are mixed experiences. You can play with mobile, desktop, and VR. Metaverses (like https://hyperfy.io or https://thirdroom.io/) are also mixed experiences. I think it's a great strength of 3D on the web to be able to enable WebXR experiences relatively easily. With R3F in particular, turning a regular Three.js scene into an XR scene is as simple as adding <XR> so I suspect there will be other cases of mixed experiences in the R3F ecosystem.

Here's how I see the next steps for XR support:

1. No-op

No-op to not break the regular desktop editor for mixed experiences scenes.

2. Enter XR with no controls

Might be easy to get it to work by simply adding R3F <XR>. This will put the user's body at (0,0,0) and they won't be able to move their body, but moving the head in VR or headset AR, and moving the phone in mobile AR should work by default to move the camera.

This is very limited, but can already be a bit useful in some situations where the scene is built at the right scale and near (0,0,0). For instance, if you are making a real-size chessboard that's positioned at table level, you can see the whole scene only by moving your "head". That's a good start.

Note that WebXR requires HTTPS even on localhost.

3. Fly controls to moving around the scene in XR

For VR and headset AR:

• Left thumbstick: Forward-Backward in the direction of the camera. Left-Right to strafe.
• Right thumbstick: Left-Right snap turning or smooth turning, Up-Down to move vertically.

For mobile AR, the same controls with virtual joysticks.

This is already a lot more useful to look around a bigger scene.

4. Editing workflow in XR

Being able to edit the scene end-to-end within XR.

For VR and headset AR:

• Selecting files with a menu (a panel that shows above the wrist for instance)
• Selecting objects via react-xr <Interactive>
• Transforming objects via react-xr <Interactive> on their gizmos
• Saving with a button on the controller or on the virtual panel.

For mobile AR:

Probably not worth implementing, but I guess it would be a mix of the XR selection / transformations above + a responsive layout to show the files and context panel outside of the canvas.

[itsdouges]

By @verekia:

Userland no-op workaround:

contexts.ts

import { createContext, useContext } from 'react'

const isInReactXRContext = createContext(false)
export const IsInReactXRProvider = isInReactXRContext.Provider
export const useIsInReactXR = () => useContext(isInReactXRContext)

canvas.tsx

<Canvas>
  <XR>
    <IsInReactXRProvider value={true}>
      <YourApp />

my-react-xr.ts

import { InteractiveProps, Interactive as XRInteractive } from '@react-three/xr'

import { useIsInReactXR } from './contexts'

export const Interactive = (props: InteractiveProps) => {
  const isInReactXR = useIsInReactXR()

  if (isInReactXR) {
    return <XRInteractive {...props} />
  }

  return <>{props.children}</>
}

// Implement other react-xr components that you use

In your app, replace:

import { Interactive } from '@react-three/xr'

by:

import { Interactive } from './my-react-xr'

[CodyJasonBennett]

Can <XR /> not be mounted in user-land? I'd be happy to export the context, but I'm confused as to the need.

[itsdouges]

There's a larger story of how best to handle missing context providers. As Triplex can open leaf components (similar to how Storybook works) you can open components that need context that aren't currently available.

Storybook handles this by adding decorators.

In @verekia'a case he also has components that are used in XR and non-XR cases.

Currently I'm positioning Triplex to not need unique "story" files so this needs a bit of thinking how best to handle.

[itsdouges]

A provider config will be available in the next release.