feature request: Location Based AR

[marcusx2]

Something like this.

[hiukim]

I personally have never worked on location based AR. but I'm open to this.

Is there particular anything you think ar.js need to improve on location based AR?

[raywu]

AR.js has done a okay job putting Aframe entity pinned to a latlng. But adding any position tag would override the latlng. I’m interested to try something else out, too

[marcusx2]

Is there particular anything you think ar.js need to improve on location based AR?

No, not really. My idea here is to have your library be a complete replacement for arjs. If I someday need image tracking and locationar, I can just use your library instead of mixing with arjs. Your image tracking is already better than theirs by a mile.

[marcusx2]

@hiukim Come on, please implement geolocation ar. You can ignore this feature request because as @akhrarovsaid points out, it can already be done with image tracking. But geolocation is another story. It's the only feature mindar doesn't have when comparing with arjs. Pretty please implement this.

In fact the time has come where I need geolocationar and image tracking on the same project. I hope it's not much of a pain to use mindarjs with arjs for this. Otherwise I'll just stick with arjs, even though your mindarjs is better!

[hiukim]

@marcusx2 I guess I could try to look into it. but I've never used that before and I don't really have much idea about it.

I guess AR.js use gps location and gyroscope? anything else?

since there is no ground tracking in ar.js, so the augmented objects is just floating in the air somehow? Is it very unstable though?

[marcusx2]

That's all as far as I know. You can read more about it here. You can try this codepen.

[raywu]

@hiukim if you're familiar with three.js there's a three.js implementation without A-Frame that could be a good reference. https://github.com/AR-js-org/AR.js/tree/dev/three.js

[hiukim]

@raywu Nice! yes, I'm familiar with three.js. Thanks for the reference!

[marcusx2]

@hiukim Is there an eta for this feature? Just curious no pressure haha.

[hiukim]

@marcusx2 sorry, was busy these days. I guess I will try to look into it on December.

[marcusx2]

Hi @hiukim , are you looking into this feature? No pressure haha, just want to know if I can expect it soon. Thanks!

[marcusx2]

@hiukim ? ;_;

[hiukim]

@marcusx2 sorry about the late reply. Actually, I've taken a little look into the location AR in ar.js.

At this moment, I have a general idea on how it works. However, I don't have much idea on what's lacking there and what to improve. Unlike image tracking, I have been using it a lot and I understand deeply what's lacking so I have strong opinion on how to improve it.

In contrast, I feel like I'm not very knowledgeable in location AR. Other than just copying the code here and update the API to make it more consistent with mindAR, I don't have much idea on what else to do.

[marcusx2]

Hi @hiukim , if you can just add the same location AR that is already present in arjs that would be great. So I don't need to mix arjs with mindarjs when image tracking and location ar are both needed.

[hiukim]

@marcusx2

I just released a new version with the support of three.js integration now instead of AFRAME. here is an example: https://hiukim.github.io/mind-ar-js-doc/more-examples/threejs-image

With this new version, I think it's easy to import both libraries. I haven't tried it, but by looking at this example: https://github.com/AR-js-org/AR.js/blob/dev/three.js/location-based/example/index.html

I can see that with the previous AFRAME version, things could get messy. but I think it's kind of straight forward now. The only thing is that there might be different versions of THREE.js

[nicolocarpignoli]

Hi everybody, I think @nickw1 might help you @hiukim on porting location based of AR.js into your project. As for now, though, AR.js has location based only on the A-Frame side.

@hiukim in location based everything is estimated based on gyroscope data and gps data. It is not very precise, but works okay for distant objects. Even if I started the feature myself, Nick is far more expert than me in general and has worked a lot with that in the past months, so he is willing to, he can probably help. I can help you anyway, but I feel like I can't do much more - in fact, all I was knowing I already put it on the docs and on the code comments, just to prepare for this moments :)

[hiukim]

@nicolocarpignoli Thx for your feedback!

I briefly looked at the new version of the location based tracking in AR.js (still in the ‘dev’ branch) I think it’s pretty nice! Well done @nickw1

I guess it really depends on whether you guys want to merge in. If you guys are open to the idea of uniting everything and collaborate on a single repo, then we can start discussing the porting process. Personally, I’d definitely love to see this happen :)

Otherwise, I don’t want to just “steal” the code from AR.js. I also don’t see a strong reason to duplicate the same piece of code. I’m not very comfortable maintaining the location part myself at this moment anyway.

Another option I can think of is to keep both repo and mutual advertise (if you are open to this). We can direct people to AR.js for marker tracking (non NFT) and geo location tracking. And MindAR for image tracking and face tracking.

[marcusx2]

I guess it really depends on whether you guys want to merge in. If you guys are open to the idea of uniting everything and collaborate on a single repo, then we can start discussing the porting process. Personally, I’d definitely love to see this happen :)

Uniting mindar with arjs? This is a good idea as well.

Another option I can think of is to keep both repo and mutual advertise (if you are open to this). We can direct people to AR.js for marker tracking (non NFT) and geo location tracking. And MindAR for image tracking and face tracking.

Nah man. I'd rather see feature parity between both libraries or merge them.

Otherwise, I don’t want to just “steal” the code from AR.js. I also don’t see a strong reason to duplicate the same piece of code. I’m not very comfortable maintaining the location part myself at this moment anyway.

Hm I understand this point.

[kalwalt]

@nicolocarpignoli threejs location-based exist in dev branch, but i have to update the webpack config file.

[nickw1]

@hiukim thanks! happy to add the location-based AR from AR.js dev into MindAR. it's the three.js based code in dev you're referring to, I presume?

In terms of the general issues with location-based AR: as @nicolocarpignoli says there are accuracy issues when you are close to the objects, due to the limitations on GPS accuracy. Also the device orientation API is a little inconsistent between browsers: Firefox does not appear to support absolute orientation (bearing relative to north) at all, and iOS/Safari uses a different API to Chrome. Chrome on Android is the most reliable. I don't have any access to iOS devices, so do note that support is likely to be limited initially to Chrome on Android.

[marcusx2]

@nickw1 I don't know about location-based AR but I am familiar with the device orientation api for all browsers. I've tested it extensively for other projects. I can tell you that it works the same across browsers, except that to activate on Safari you need to request permission on a user gesture.

[nickw1]

@marcusx2 thanks. I am referring specifically to absolute device orientation, i.e. being able to obtain a compass heading relative to north. On Chrome you need to handle the deviceorientationabsolute event for this, rather than deviceorientation. On Safari you use webkitCompassHeading. On Firefox, from what I can make out, you cannot obtain a compass heading relative to north at all. (Ironically it looks like this used to be possible, but isn't now). The ordinary deviceorientation event appears to give an alpha value which is relative to where the device was when the application was first launched, rather than relative to north - which is of limited use in a location-based AR app.

All this is to the best of my knowledge, and what I have researched so far: I may be wrong, so if you are aware of a way to obtain the compass heading relative to north on all three browsers, do let me know.

[nicolocarpignoli]

@hiukim @nickw1

Let's see from a user perspective: I'd like to have some kind of OSS Web AR technologies for my project. I have two choice: go with one library that has all of them, or merge more libraries (and learn them separately).

As for now, AR.js cannot entirely replace mindARjs, nor mindARjs can replace totally AR.js.

I think that we should go for gradual steps. We can list in both project's readme (or docs) which features are supported, and redirect for other features to the other library, so:

• Marker Based => AR.js
• Image tracking => mindAR (it is not perfect but absolutely better than AR.js one)
• Location Based => AR.js (for now)
• Face Tracking => mindAR

@hiukim if you're okay with that, I can edit the AR.js readme in zero time. I think that we, as maintainers/contributors, should make the way easy for "old" users and newcomers, that's the most important thing (also to attract new contributors).

[hiukim]

@nicolocarpignoli I'd love that. I totally agree with what you said. Gradual steps make perfect sense. In my opinion, attracting new contributors in OSS is a very important goal.

In this case, I think @nickw1 can continue working on the AR.js repo. It might be easier for him to focus only on the location based features. We can see how to align the library APIs so users don't have to relearn everything from ground. I might be able to help with some blog posts or tutorials later.

[hiukim]

@nicolocarpignoli I went ahead to add AR.js on top of the README. I am also thinking about a better introduction to make the community more united. Let me know if you have better idea and I'm happy to change the wordings.

[nicolocarpignoli]

@hiukim added mindAr references on top of AR.js readme too! Maybe a blog post (on personal blog, or medium, dev.to anywhere) might be a good reference to link? Just an idea. I can do that, but it might take me some week. Just a simple one saying: "this is the current state of the art for free oss web ar: for markerless, go with model viewer, for face tracking, go with mindAr... and so on".

[nickw1]

@nicolocarpignoli @hiukim yes sounds good to me, happy to focus on AR.js, agree about not fragmenting what is out there.

[marcusx2]

So mindarjs won't have location based ar? ;_;

[hiukim]

seems not for now :(

[marcusx2]

I hope ARJS and mindarjs gets merged together at some point, to make it easier to use features from both on the same project.