search

hiukim / mind-ar-js

Web Augmented Reality. Image Tracking, Face Tracking. Tensorflow.js
MIT License
2.28k stars 426 forks source link

feature request: marker based ar #55

Closed marcusx2 closed 2 years ago

marcusx2 commented 3 years ago

Something like this.

marcusx2 commented 3 years ago

@hiukim do you intend to add all the features present on arjs? That would be nice, to make mindarjs a complete alternative solution to arjs.

hiukim commented 3 years ago

@marcusx2 Just wondering do people still use this kind of markers given that there is natural image tracking?

At this moment, I personally have no plan to work on this type of "marker based". But I'm open to it. If a lot of people think it's useful, or if anyone interested to implement it, I'm happy to discuss.

marcusx2 commented 3 years ago

@hiukim Yes, people use it. Think of it as normal image tracking, but it's a generic image that goes on top of several other images. It's used when you want to activate AR content with any image. A big qrcode is placed on top of the image(imagine there are many image targets) and is used for activation and tracking at the same time. So instead of tracking several different image targets, you have just one qr code.

Anyhow, I'd love to see mindarjs as a complete replacement for arjs.

What about this feature request?

akhrarovsaid commented 3 years ago

@marcusx2 NFT (Natural Feature Tracking) is capable of tracking arbitrary images. You can upload your ArUco marker, QR code, or other marker to the compiler tool and achieve pretty much the same results.

I'd, personally, be much more interested in having 'Plugins' for MindAR that could switch between backends - say instead of the FREAK detector/descriptor we can swap it out for ORB(v1/v2/v3). Even more interesting is if @hiukim has explored some recent advancements in employing a stronger ML presence in the library, for example this paper where the authors compute the homography parameters beyond just a 4-point bounding box. Could be useful for world-tracking / image-tracking by computing the homography between two adjacent frames and reconstructing a camera pose.

Blackclaws commented 3 years ago

The problem is that normal image tracking is much less stable especially on longer distances. If you have a multitude of markers that you want to use to create content at the same time NFT is rather slow. Markers can usually be read over a longer distance.

marcusx2 commented 3 years ago

So then let's have marker based ar as well! Hahaha

nicolocarpignoli commented 2 years ago

In my opinion, I agree with @Blackclaws and markers still have their own sense for performances and tracking stability, even if Mind-arjs image tracking is already great. Anyway, I do not know if this is the best feature where to carry efforts: I mean, what are the most used type of AR today?

I think (personally) that markers are a subset of image tracking, with some enhancements but also bigger limitations. If I'd have to choose, I will not go for markers as first next thing to add but rather on Markerless. Just my opinion looking at the current market situations and other paid libraries like 8thWall.