USDZ in AR QuickLook flies on camera level, does not acknowledge the world/camera

[Rubenvdveen]

Describe the bug

I’m trying to use QuickLook AR to show the 3D ThreeJS model in AR on iOS. Exporting my 3D object to USDZ using the USDZ exporter works fine. When opening the USDZ file in XCode everything seems to be fine.

Though, when opening the USDZ in QuickLook AR, the 3D model is flying above the ground, on my camera’s Y level. The camera PoV is positioned exactly in the middle of the X and Z axis of the 3D model and at the bottom of the Y level.

I have another problem with opening the USDZ in QuickLook AR, which is; When opening the USDZ in QuickLook AR, the model is invisible at first. Then when I scale the model down to < 10%, the model becomes visible, though it does not scale in size at all. Also, the “Model” tab in QuickLook does not even show the 3D model. When switching between the “Model” and “AR” tabs, the model flies by really quick.

To Reproduce

For reference, I’ve added my USDZ model below.

Button click code:

newScene.add(sceneRef)
              const pivot = new THREE.Object3D()
              newScene.add(pivot)
              pivot.add(sceneRef)
              // position the object on the pivot, so that it appears 5 meters
              // in front of the user.
              pivot.position.z = -50

              const yaxis = new THREE.Vector3(0, 1, 0)
              const zaxis = new THREE.Vector3(0, 0, 1)
              const direction = zaxis.clone()
              // Apply the camera's quaternion onto the unit vector of one of the axes
              // of our desired rotation plane (the z axis of the xz plane, in this case).
              direction.applyQuaternion(cameraRef.quaternion)
              // Project the direction vector onto the y axis to get the y component
              // of the direction.
              const ycomponent = yaxis
                .clone()
                .multiplyScalar(direction.dot(yaxis))
              // Subtract the y component from the direction vector so that we are
              // left with the x and z components.
              direction.sub(ycomponent)
              // Normalize the direction into a unit vector again.
              direction.normalize()
              // Set the pivot's quaternion to the rotation required to get from the z axis
              // to the xz component of the camera's direction.
              pivot.quaternion.setFromUnitVectors(zaxis, direction)
              // Finally, set the pivot's position as well, so that it follows the camera.
              newScene.getWorldPosition(cameraRef.position)

              newScene.updateMatrixWorld(true)
              iosExporter.parse(newScene).then((result) => {
                saveUSDZString(result, 'scene.usdz')
              })

saveUSDZString function:

  function saveString(text: any, filename: any) {
    save(new Blob([text], { type: 'application/json' }), filename)
  }

save function:

  function save(blob: any, filename: any) {
      link.href = URL.createObjectURL(blob)
      link.download = filename
      link.rel = 'ar'
      let img = document.createElement('img')
      img.alt = 'hi'
      img.src = 'https://google.com/img'
      link.appendChild(img)
      link.click()
  }

USDZ model: scene (43).usdz.zip

Expected behavior

What I’m trying to accomplish is to position the 3D model in front of me, and for the 3D model to acknowledge the world shown by the camera. The 3D model should stick to walls, or at least the floor to begin with.

Platform:

• Device: Mobile
• OS: iOS
• Browser: Safari
• Three.js version: 0.138.3

[mrdoob]

Exporting my 3D object to USDZ using the USDZ exporter works fine.

Can you share the file you're converting to USDZ?

[Rubenvdveen]

@mrdoob In what format do you want it? Before converting it's a ThreeJS scene, built from several .obj files and .png's.

On another forum, someone suggested that I check the scale of the USDZ. He noticed it was x1000 for almost all scales, which could be the reason why QuickLook does not show the model correctly. I'm going to do that somewhere in the upcoming days, I'll keep you posted here.