XRLayer invisible when background color or fog is used

[appbisweb]

When I tried to implement XRLayer into an existing R3F app, the images and videos disappeared. So I made a simple test setup.

What happened? Images and videos placed with XRLayer were invisible in the Meta Quest 3. It is working in the Immersive Web Emulator in Chrome.

To make sure they weren't just in the wrong spot, I tried it with Drei Image, which worked as expected.

Removing Background Color "helped" <color attach="background" args={['#ededed']} />

I figured out that the following line was causing this problem. Is this a bug of XR, the Meta Quest Browser or am I doing something wrong?

Dependencie:s

• react-three/drei: 9.111.4
• fiber: 8.17.6
• react-three/xr: 6.2.2
• react: 8.3.1

Test Code:

  const toggleVR = () => {
    store.enterVR()
  }

  const video = useMemo(() => {
    const result = document.createElement('video')
    result.src = 'long.mp4'
    return result
  }, [])

  const image = useMemo(() => {
    const result = document.createElement('img')
    result.src = 'test-img.jpg'
    return result
  }, [])

  return (
    <>
      <button onClick={toggleVR}>Enter VR</button>
      <Canvas>
        <color attach="background" args={['#ededed']} />
        <OrbitControls enabled={false} />
        <XR store={store}>
          <XRLayer
            position={[-0.3, 1.5, -2.5]}
            onClick={() => video.play()}
            scale={[2, 1, 1]}
            src={video}
          />
          <Image
            url={'test-img.jpg'}
            position={[-1.2, 0, -2.5]}
            scale={[2, 1, 1]}
          />
          <XRLayer scale={[2, 1, 1]} position={[1.2, 0, -2.5]} src={image} />
        </XR>
      </Canvas>
    </>
  )
}

export default App

[bbohlender]

This is a limitation of the concept of layers. Layers are punching a hole through the 3D Layer so that they can be visible when blending the 3D layer and the 2D layers together. Pushing a hole works by drawing to the depth buffer early so that everything else only draws on that part of the scene if it's actually in front. This technique cannot work with setting a background color on the scene, because that background color sets the clear color to a non-transparent color making the punsh hole technique impossible because you can only add transparency but not remove it.

The simplest solution imo is adding a very large cube with the background color.