microsoft / MixedReality-WorldLockingTools-Unity

Unity tools to provide a stable coordinate system anchored to the physical world.
https://microsoft.github.io/MixedReality-WorldLockingTools-Unity/README.html
MIT License
188 stars 45 forks source link

WorldLocking Tool in big similar environment #192

Open Manuel-Meyer opened 3 years ago

Manuel-Meyer commented 3 years ago

Hi,

I am trying to use WorldLocking tools in a large warehouse. The problem with this is that the environment is very similar and WLT occasionally thinks it's already been to one place, even though it's the first time I walk by. This causes the position, which the tracking recognizes much better, to be reset every now and then and you get a distorted image of reality. Is it possible to tell the WLT to rely more on tracking, or to check the environment more closely for similarities?

I'm trying to put a digital shelf in the hall with the help of QR codes in order to be able to mark the storage locations.

Does anyone have any experience here? Or tips on how to get the problem under control? I would be grateful for any help.

Many greetings Manuel

zantiu commented 3 years ago

Do you have some video or pictures of your environment? What device are you using? How big is the area where you use WLT? And how far are the position jumps?

We've been using Hololens 2 in some pretty repetitive environments, although maybe not so repetitive as a warehouse where every rack could almost look the same.

I think now all anchors keep tracking the environment. So even if you walk 20 meters away the anchor on that position will try to recognize the position (or some anchormanager tries to tie the position to this anchor, and all anchors that are active)

I'm not sure what's going on under the hood, but maybe once a certain confidence has been reached about a location, then anchors that are far away (for example > 10 meter) should stop trying to lock onto the environment, because the anchors near to you will be more relevant. (It would also as an additional bonus help with the performance in big areas.)

But I'm not sure if there's a practical solution for this at this moment. I don't think anchors can be suspended temporary, and storing them and retrieving later is also not an option in many cases.

It's possible that the underlying platform/device already tries to do something in this regards, but I'm not aware of it.

Of course when the tracking is lost then there's not much else you can do then to use all the anchors, unless there's some other info that can help give a rough position indication.

Manuel-Meyer commented 3 years ago

Hi zantiu,

thanks for the quick reply. I also use the Hololens 2. The part of the warehouse I use is about 50 by 50 meters and the position jumps every now and then to the last anchor.

I'm just wondering if more anchors would help me, however they would also be more computationally intensive. I have so far simply taken the default setting 1m anchor distance and have increased this slightly to simplify the calculation. Does it benefit if I add more QR codes to the warehouse? Then it could orient itself more to these.

Another problem is of course that the environment can change every now and then, for example when a pallet is taken from the shelf. If the tracking is lost, I can scan one or two QR codes again. That should not be a problem.

zantiu commented 3 years ago

What do you mean with "to the last anchor"? What is the last anchor?

I don't think more anchors will help. Adding more QR codes will not impact this.

The environment changing is another problem. I'm thinking that probably there's a way to automatically mark anchors as obsolete if for example all the surrounding nearest anchors are actively being tracked but not the obsolete one. In that case the anchor could be removed and a new one created.

Manuel-Meyer commented 3 years ago

I have the impression that it jumps to the last generated anchor. So if I walk down a aisle and set an anchor every meter, it jumps back to the anchor 1 meter ago. I can unfortunately only check this again tomorrow in more detail, but I'll report back here then. I also have the fear that it is possibly just a bit too dark in the warehouse. I will check that, too.

Manuel-Meyer commented 3 years ago

I have tested the whole thing again and have come to the conclusion that I do not need the anchors. I set the distance to 99999 (min and max) for this and rely entirely on the scanned QR codes. i have one in each aisle at the start and at the end. So I can determine the position of the shelf exactly enough for my purposes and in case of tracking failures I have to go back to the start of the aisle. At the start and end of the aisle the environment is unique enough for WLT and there are no misalignments.

Thanks for your help! Many greetings Manuel

zantiu commented 3 years ago

Interesting. Curious what @fast-slow-still 's thoughts are on this approach?

How long are your aisles? I would expect the position error to become considerable towards the middle of the aisles, unless if they're rather short. Will you use this in production, or is it just a demo?

Manuel-Meyer commented 3 years ago

I would guess they are roundabout 50 meters long. But if I get to the end the rack is shifting towards the QR-Code at the end, so I have an error of 25 meters. If I have more problems I could add an QR-Code in the middle of the aisle. At the moment it is just a showcase.

fast-slow-still commented 3 years ago

It's an interesting approach. With the distance set so large, you will basically be relying on a single anchor (at the starting point).

Anchors become unreliable when you get more than a small number of meters away (hence the default 1 meter spacing). So you would need to keep scanning QR codes to fight drift, and as @zantiu says, you will have nothing locking the world in place in the middles of the aisles.

All of that said, each application scenario has its own requirements, so just because your approach wouldn't work for everyone doesn't mean it won't work in your situation. A way to achieve similar results would be to set the Anchor Subsystem type (in WorldLockingContext/AnchorManagement settings) to "Null". This uses no-op anchors which don't communicate with the device, and always report that they have not moved.

Another approach which I have seen successfully used is to decorate the environment to disambiguate. For instance, putting a sheet of paper with a large block letter A,B,C,... every few meters along the wall/aisle. Or a sparse ring of lights in the periphery set up in an asymmetric pattern (e.g. one lamp on one end of the aisle and two lamps on the other).

The environment you describe, low light and lack of features, is certainly the most challenging for the inside out tracking that the HoloLens uses (and Android and iOS). Ultimately WLT relies on the data received from the device. If the data has errors, WLT can use the graph of anchors to correct for it. But if the data is just flat out wrong, or there is no tracking at all, WLT can't really help.

If possible to temporarily add recognizable features to your environment, or improve the lighting, you will get improved tracking results and improved user experience all around.

Also, for an environment as large as yours, I would recommend a larger spacing between anchors, something in the range of 2-5 meters. But again, that depends so much on your environment.