Question: How to do floor, plane, surface detection and tracking?

[S-Cardenas]

Is there any guidance as to how to do world tracking with this library? I see that in the Readme.md there is a reference to "World Tracking", but I don't see any examples that use it.

If this is a new feature that needs to be added, I would gladly look into it. I imagine we could use Tensorflow.js to try and accomplish this task.

Any guidance or input is much appreciated.

~ Cheers

[hiukim]

This library doesn't provide any world tracking, but it's definitely something very interesting and useful to add.

Welcome any ideas/advices/contributions in this area.

[S-Cardenas]

Thanks for the quick response @hiukim. I'm looking to add something like this room / floor tracker that you see in the WayFair App. Are there any Tensorflow.js resources that someone in this community can point me to so I can build it out and try to add it to this library?

https://user-images.githubusercontent.com/14102289/151337911-8fefafd3-2bbf-40c5-bbbe-d6df2986f0c4.mp4

[Makio64]

@S-Cardenas you can add it via webxr API for android only but it should be support in the futur by safari/ios as well.

[marcusx2]

I mean, once webxr is enabled by default on all browsers, mindarjs, arjs, etc will all become quickly obsolete. The edge that these libraries will hold for a while is being able to run on older devices, but as time goes on this will become less and less of a problem.

If we are talking about non-webxr solutions, Zappar and 8th Wall have their own custom solutions for world tracking. But for obvious reasons they won't share how to do it.

[benferns]

For me personally I'm looking at mind-ar-js as a way to match an iOS quicklook experience on Android. I'd like to have a way to have the (stationary) marker report its position while using webxr for the main tracking, so I update the 'anchor' of the scene sporadically with smoothed marker data.

On the broader questions, I think there will be a few years (at a minimum!) before there isn't a need for shim-like libraries to bridge these kinds of functionality. Chrome moves fast in WebXR, but some Chinese android browsers and webview configurations lag that progress.

Safari devs have shown interest, but only in HMD-related areas from what it seems (the original WebVR use-case). They've had half a decade to implement handheld WebXR but didn't, so it wouldn't be out of character to continue to ignore the use-case in favor of pushing a singular OS-spanning UX based around USDZs and their RealityOS.

All this is a long-winded way of asking if anyone has tried using this library in combination with WebXR, and adding a +1 to the idea of an android-only world-tracking feature that integrates with marker positions!

[FlorianGauvrit]

For me personally I'm looking at mind-ar-js as a way to match an iOS quicklook experience on Android. I'd like to have a way to have the (stationary) marker report its position while using webxr for the main tracking, so I update the 'anchor' of the scene sporadically with smoothed marker data.

On the broader questions, I think there will be a few years (at a minimum!) before there isn't a need for shim-like libraries to bridge these kinds of functionality. Chrome moves fast in WebXR, but some Chinese android browsers and webview configurations lag that progress.

Safari devs have shown interest, but only in HMD-related areas from what it seems (the original WebVR use-case). They've had half a decade to implement handheld WebXR but didn't, so it wouldn't be out of character to continue to ignore the use-case in favor of pushing a singular OS-spanning UX based around USDZs and their RealityOS.

All this is a long-winded way of asking if anyone has tried using this library in combination with WebXR, and adding a +1 to the idea of an android-only world-tracking feature that integrates with marker positions!

I am currently trying to use MindAR along with WebXR. I detect my marker and then I open a WebXR session in which I display my AR scene.

I have some difficulties positionning my object as I don't have the coordinates of my marker. Could you tell me more about your 'anchor updating' solution?

[marcusx2]

@hiukim Closed as completed? So mindar js supports surface detection and tracking?

[hiukim]

@marcusx2 sorry, nope. I was just trying to clean up some old issues. It's been stale for a while and there is no plan for that at the moment.