search

google-ar / arcore-android-sdk

ARCore SDK for Android Studio
https://developers.google.com/ar
Other
4.95k stars 1.22k forks source link

Office scenes containing recurring elements cause positioning errors when similar images are captured #1118

Open mode51software opened 4 years ago

mode51software commented 4 years ago

SPECIFIC ISSUE ENCOUNTERED

Offices tend to have elements in the scene that recur frequently. This can cause the positioning to jump to a previous element, to some extent possibly because the images look identical.

It would be useful to be able to hint to ARCore that the scene is a home, an office, a shopping centre etc. Then weight internal parameters accordingly. For example, in an office, rely less on image processing and allow dead reckoning to override instances where the image looks identical to a previous one.

As I have no insight into ARCore's internals, my assumptions could well be wrong. The positioning errors do appear to be repeatable though and at precisely the same points.

VERSIONS USED

Packages: versionName=1.19.202100803 Hidden system packages: versionName=0 Active APEX packages: Inactive APEX packages: Factory APEX packages:

STEPS TO REPRODUCE THE ISSUE

  1. Find an office with lots of recurring infrastructure within the scenes. The office used here is the Bradfield Centre, Cambridge Science Park, CB4 0GA, UK

  2. Use ARCore's motion tracking and observe how the positioning is affected by recurring elements in the scenes.

During a walk test, this image was captured:

image

Then, a minute later, this similar image resulted in the position being set back to the location of the previous one:

image

I also had (repeatable) issues with the following scene on a different day where I thought perhaps reflections on the floors could be an issue or something to do with the pillars. I note the capture above was also near to a pillar.

image

Observe this external scene where recurring elements of the building result in the tracking going off:

image

WORKAROUNDS (IF ANY)

ADDITIONAL COMMENTS

devbridie commented 4 years ago

Hey, thanks for your detailed report. I think I understand the problem, but could you elaborate on how the screenshots show that the tracking isn't functioning as expected? I don't quite understand the relation between the images and the behavior.

mode51software commented 4 years ago

Thanks Dereck,

In the smaller screenshot at the top the first one was at 4m 43s into the survey, the second one at 5m 51s. The position in 3D space reset at 5m51s back to where I was at 4m43s. I'm speculating that the very similar image in the camera view, which is what ARCore was ingesting, may relate to this. I seem to find a small number of specific locations in the office building where ARCore keeps repeating this type of behaviour.

The issue relates to how ARCore positioning attempts to recover I think. I recall in Tango there was a DRIFT_CORRECTION parameter so I guess it's along these lines. In this office case ARCore seems to be too aggressive in that it is recovering when the motion tracking hasn't gone off as such, ie. if it tried less hard to recover and just kept going then it would fine. I note that ARCore does already work remarkably well (with the additional ToF input), just I think there's scope for more tuning based on the type of environment.

I'll email you a link to a video of the survey that the screenshots were taken from.

mode51software commented 4 years ago

I note the original survey was done with just horizontal plane detection enabled. I've now enabled vertical planes as well though in the following video I'm not drawing them on the camera preview. Seemed to work better this time and with the latest 1.19 release of ARCore:

https://live.signalscore.co.uk/downloads/sigsite-bradfieldground-20200914.mp4

roehroeh commented 2 years ago

Hi Sorry to perform necromancy on this topic, but has there been a development on the issue itself? I have an extremely repetitive environment and have noticed many such relocations (running on 1.29, can't use anything newer right now) despite enabling all planes. The relative movement to prior poses (saved as session anchors and updated) I extract from consecutive frames suffer greatly from this, as the correction/relocation seems to create large jumps towards some anchors not in view.