search

google-ar / arcore-android-sdk

ARCore SDK for Android Studio
https://developers.google.com/ar
Other
4.98k stars 1.22k forks source link

Pose.getTranslation() not working on OnePlus Nord N10 5G #1217

Open henrikyn opened 3 years ago

henrikyn commented 3 years ago

SPECIFIC ISSUE ENCOUNTERED

We had a Firebase logs from a case where user is unable to get ARCore tracking the translation on OnePlus Nord N10 5G. camera.pose.translation returns a FloatArray where all values are always 0f.

We check if the translation is returning correct values once every second, and if all the values are 0s for every check for 5 seconds we determine that the translation is not working properly and finish the Activity using ARCore. This happened to the user for 12 times.

VERSIONS USED

STEPS TO REPRODUCE THE ISSUE

WORKAROUNDS (IF ANY)

ADDITIONAL COMMENTS

devbridie commented 3 years ago

Are you checking [camera.trackingState](https://developers.google.com/ar/reference/java/com/google/ar/core/Camera#getTrackingState()) prior to retrieving the camera pose? From the documentation:

Returns the current motion tracking state of this camera. If this state is anything other than TrackingState.TRACKING the pose should not be considered useful.

henrikyn commented 3 years ago

We're not checking the TrackingState because we only start to check the translation when the user starts recording. While recording there's status borders displayed on the UI for the user. Orange borders are displayed only when TrackingFailureReason is INSUFFICIENT_LIGHT, EXCESSIVE_MOTION or INSUFFICIENT_FEATURES. And green borders are displayed on start and when the TrackingState has is TRACKING again after an previously displayed orange borders. User sent a video of this issue and it has green borders the whole time.

From my testing the translation is only [0f, 0f, 0f] if TrackingState has never been TRACKING. But if it has been TRACKING even once it will give the previous translation value once TrackingState is PAUSED (e.g. -0.027799891, -0.013697374, -0.07906155).

This means that the TrackingFailureReason could be BAD_STATE or CAMERA_UNAVAILABLE or TrackingState could be STOPPED or PAUSED from start though. And it would stay that way for the whole time and ARCore would never start TRACKING.

Only way to reproduce this with the devices I have is to start ARCore session and where the camera is looking at a table top the whole time and sees nothing, then the translation will give me [0f, 0f, 0f] and throw this error after 5 seconds or recording. I have not tested with OnePlus Nord N10 5G because I don't have one. We have thousands of users and this rarely happens, because ARCore usually starts tracking before the user starts recording and if not it usually starts tracking during first 5 seconds of recording.

In this case user had 12 ARCore sessions where the duration of the session was between 8-20 seconds and then cut out because the translation was only zeros for 5 seconds once started recording. So ARCore didn't start tracking for 20 seconds or the translation was not returning correct values.