search

google-ar / arcore-android-sdk

ARCore SDK for Android Studio
https://developers.google.com/ar
Other
4.91k stars 1.2k forks source link

Incorrect depth data #1579

Open marta-opalinska opened 1 year ago

marta-opalinska commented 1 year ago

SPECIFIC ISSUE ENCOUNTERED

The depth data seems to be incorrect during some sessions - some of the anchors are placed in a completely different plane in 3D space than they should be. I've created an app illustrating the issue by creating anchors around a circle. Then for each point, I'm saving Pose tx, ty, ty in a JSON file for further analysis. With the attached Python script, I'm plotting the hit points as well as points in 3D space - red dots are the points I was hitting (plot 1) and Plot 2 shows what poses were returned by ARCore in 3D space:

image image

It is clear that half of the circle is in a different plane. I've tried it many times and this happens frequently enough to be observable. The result is not material or shape dependant - I've tried with non-reflective plastic and square objects.

VERSIONS USED

corrupted_depth_data.zip

In the attached zip file you can find:

The code to a sample app can be found here: https://github.com/marta-opalinska/arcore-test-app

STEPS TO REPRODUCE THE ISSUE

  1. Run the code sample app and point that to any object less than 15 cm in diameter to try to measure the circle.

WORKAROUNDS (IF ANY)

No

ADDITIONAL COMMENTS

CyberJacks commented 1 year ago

Hello,

I am new tie-wearing #mattgroening

Jk.

I am new to programming/coding/SDKs/GitHub and am just curious as to see if this will elicit a response.

Thank you for using the formatted template - it helps my future understanding.

Sincerely, Jackson

On Thu, 27 Apr 2023, 9:11 pm Marta Opalinska, @.***> wrote:

SPECIFIC ISSUE ENCOUNTERED

The depth data seems to be incorrect during some sessions - some of the anchors are placed in a completely different plane in 3D space than they should be. I've created an app illustrating the issue by creating anchors around a circle. Then for each point, I'm saving Pose tx, ty, ty in a JSON file for further analysis. With the attached Python script, I'm plotting the hit points as well as points in 3D space - red dots are the points I was hitting (plot 1) and Plot 2 shows what poses were returned by ARCore in 3D space:

[image: image] https://user-images.githubusercontent.com/50515248/234860800-2ba00697-60d7-4b99-b1de-399be980ef5f.png [image: image] https://user-images.githubusercontent.com/50515248/234862369-ba8f89e3-6f60-432a-8bc6-f43b9bd59f9d.png

It is clear that half of the circle is in a different plane. I've tried it many times and this happens frequently enough to be observable. The result is not material or shape dependant - I've tried with non-reflective plastic and square objects. VERSIONS USED

  • Android Studio: 11.0.15+0-b2043.56-9505619 amd64
  • ARCore SDK for Android: 1.36.0
  • Device manufacturer, model, and O/S: Samsung S10 5G, Android 12
  • Google Play Services for AR (ARCore): 1.36.230390493 On Windows, use: adb shell pm dump com.google.ar.core | findstr /i "packages: versionName" On macOS, use: adb shell pm dump com.google.ar.core | egrep -i versionName|packages:
  • Output of adb shell getprop ro.build.fingerprint: amsung/beyondxeea/beyondx:12/SP1A.210812.016/G977BXXSEHWC1:user/release-keys

corrupted_depth_data.zip https://github.com/google-ar/arcore-android-sdk/files/11343647/corrupted_depth_data.zip

In the attached zip file you can find:

  • dataset file containing a corrupted sample - JSON files with depth data, confidence data etc., photos (I can provide more of these if necessary)
  • circle_analysis.ipynb - jupiter notebook file with circle analysis including additional graphs

The code to a sample app can be found here: https://github.com/marta-opalinska/arcore-test-app STEPS TO REPRODUCE THE ISSUE

  1. Run the code sample app and point that to any object less than 15 cm in diameter to try to measure the circle.

WORKAROUNDS (IF ANY)

No ADDITIONAL COMMENTS

— Reply to this email directly, view it on GitHub https://github.com/google-ar/arcore-android-sdk/issues/1579, or unsubscribe https://github.com/notifications/unsubscribe-auth/AK3NZUOT7XINDJ3LDBROG63XDJWAVANCNFSM6AAAAAAXN27YVY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- This message (and any associated files) may contain confidential and/or privileged information. If you are not the intended recipient or authorized to receive this for the intended recipient, you must not use, copy, disclose or take any action based on this message or any information herein. If you have received this message in error, please advise the sender immediately by sending a reply e-mail and delete this message. Thank you for your cooperation.

marta-opalinska commented 1 year ago

The issue remains in the updated version of ARCore 1.37.0.

raju535482 commented 1 year ago

+1 for this issue

raju535482 commented 8 months ago

@devbridie can you please tell us, if there is any scope to improve depth data/image quality with ToF sensor in ARCore ?

Because, Samsung's 3D scanner app with help of S20 Ultra ToF sensor, is able to produce better 3d reconstruction, and the same Samsung S20 Ultra device is struggling to get good depth quality in ARCore.