Using ViroARCamera (to fix objects relative to the screen) causes ViroARImageMarker child objects to be positioned incorrectly

[jacklj]

• [x] Review the documentation: https://docs.viromedia.com/
• [x] Search for existing issues: https://github.com/viromedia/viro
• [x] Use the latest ViroReact release: https://docs.viromedia.com/docs/releases

Environment

  React Native Environment Info:
    System:
      OS: macOS 10.14.6
      CPU: (4) x64 Intel(R) Core(TM) i5-6267U CPU @ 2.90GHz
      Memory: 473.89 MB / 16.00 GB
      Shell: 5.3 - /bin/zsh
    Binaries:
      Node: 10.18.0 - ~/.nvm/versions/node/v10.18.0/bin/node
      Yarn: 1.21.1 - /usr/local/bin/yarn
      npm: 6.13.4 - ~/.nvm/versions/node/v10.18.0/bin/npm
      Watchman: 4.9.0 - /usr/local/bin/watchman
    SDKs:
      iOS SDK:
        Platforms: iOS 13.2, DriverKit 19.0, macOS 10.15, tvOS 13.2, watchOS 6.1
      Android SDK:
        API Levels: 23, 24, 25, 26, 27, 28
        Build Tools: 19.1.0, 20.0.0, 21.1.2, 22.0.1, 23.0.1, 23.0.2, 23.0.3, 25.0.0, 25.0.1, 25.0.2, 26.0.1, 26.0.2, 27.0.3, 28.0.3
        System Images: android-23 | Google APIs Intel x86 Atom, android-26 | Google APIs Intel x86 Atom, android-28 | Google Play Intel x86 Atom
    IDEs:
      Android Studio: 3.5 AI-191.8026.42.35.6010548
      Xcode: 11.2.1/11B500 - /usr/bin/xcodebuild
    npmPackages:
      react: 16.8.3 => 16.8.3
      react-native: 0.59.9 => 0.59.9

I'm seeing this bug on iPhone SE (iOS 13), both when running the app via the Viro Media app, and when building it from Xcode.

"react-viro": "2.17.0",

Description

When I add a <ViroARCamera /> to my ViroARScene, it causes objects to shift slightly in position. It causes objects which are children of ViroARImageMarkers to shift much more.

This is even the case when I explicitly set the ViroARCamera's position and rotation to be [0,0,0] (which is, as far as I know, the default). Therefore I'm not sure why it's causing objects to jump to new locations.

I want to use ViroARCamera so that I can position a separate object always in the centre of view (ie not placed in the real world). This works, but then all objects positioned in the world jump to different locations. The locations they jump to seem to vary randomly in direction and distance from the marker.

Perhaps the mounting of the ViroARCamera component is causing the viewer's camera location relative to the AR objects to change by a random amount?

Reproducible Demo

I've made a video of the behaviour. At around 35 seconds in, I press a button that causes the <ViroARCamera /> component to be added, and you see the marker objects jump.

Video url: https://www.dropbox.com/s/keftpn0mn80cvb2/viro%20AR%20camera%20bug.mp4?dl=0

The code is something like this:

  render() {
    const { showArCameraView } = this.props;

    return (
      <ViroARScene>
        {showArCameraView && (
          <ViroARCamera
            position={[0, 0, 0]}
            rotation={[0, 0, 0]}
          />
        )}
            <ViroARImageMarker
              key={`marker ${markerId}`}
              target={`${markerId}`}
              onAnchorFound={...}
              onAnchorRemoved={...}
              onAnchorUpdated={...}>
                 <CrossHair />
            </ViroARImageMarker>
     ...
... 

[tomascg2309]

@jacklj hey, dude! fixing objects to armarker doesn't need a camera, why do you using that component?

[jacklj]

Hey @tomascg2309, I’m not trying to fix objects to AR markers (I’m already doing that successfully) - I’m using ViroARCamera to fix some Viro Components to always be in the same place on the screen, for a Heads-Up Display. However, when I add the ViroARCamera component (as a sibling to the ViroARImageMarkers, not as their parent), all the image marker child objects jump and aren’t directly on their markers anymore.

Thanks!

[JackSoby]

hey @jacklj I am having the same issue! did you have any luck figuring this out?

[cenaHara]

hi @JackSoby , @jacklj Have you both resolved this issue

[cenaHara]

hi @JackSoby , @jacklj HOW TO CHECK TARGET CENTER of view

[LouisOhlow]

Hey guys, i faced the same error and had a little work around which made the job: i replaced the ViroARCamera with a simple ViroNode component and set it's position and rotation to the camera's position and rotation. I retrieved those from the ViroARScene with onCameraTransformUpdate

[andrewjb123]

Hey guys, i faced the same error and had a little work around which made the job: i replaced the ViroARCamera with a simple ViroNode component and set it's position and rotation to the camera's position and rotation. I retrieved those from the ViroARScene with onCameraTransformUpdate

hi, would you be able to provide an example? im having exact same issue

[serdarcevher]

I've also noticed that as soon as I render a ViroARCamera component, ViroARScene stops returning the position values from the onCameraTransformUpdate event. The problem described above could be related to this issue.

Everything else (rotation, forward etc.) remain working but the position array starts to return as [0, 0, 0]. This is not happening when I avoid using ViroARCamera.

The workaround described by @LouisOhlow works, but it caused some delay to reposition the element in my case. Anyway for anyone who wants to try it,

1) Save the cameraTransform returned by ViroARScene into a state object, 2) Render your ViroNode as:

<ViroNode
    position={state.cameraTransform.position}
    rotation={state.cameraTransform.rotation}
>