Is Markerless AR possible with AR.js?

[MP619212]

Apple ARKit and Google ARCore allow apps to generate AR Content anchored to the real world using Plane Detection instead of Markers.

Is this possible with AR.js on iOS and Android devices?

GOAL To view AR Content on a Mobile Web Browser without using a Marker.

Thanks!

[jarodium]

https://en.wikipedia.org/wiki/LLA_Marker Different approach from what you want but I think it could be a great addon for this lib

[ajsutrave]

Hey has anyone made any progress on this? If not, I'd like to volunteer

[MP619212]

Please contact me at: mattpowell1985@gmail.com and let's discuss via Skype.

[larsgaardsoe]

I'm also very interested in this. How can I follow og maybe contribute?

[MP619212]

To be honest with you, I don’t come from a technical background; I just had an idea for a business that would involve this.

I was just asking to see if this is possible given existing technology.

So I’m not exactly sure where we would start...

Matt Powell +1(619) 791-6159 LinkedIn.com/in/mattpowell1985

On Sep 29, 2017, at 3:13 AM, larsgaardsoe notifications@github.com wrote:

I'm also very interested in this. How can I follow og maybe contribute?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jeromeetienne/AR.js/issues/190#issuecomment-333049727, or mute the thread https://github.com/notifications/unsubscribe-auth/AekW4-oHfLk3yPQYTZktBbDOesvwBzznks5snJikgaJpZM4PacoQ.

[ajsutrave]

OK so I took a little bit of a look. This might be involved change because the Daqri ARKit package that is compiled by ASM.js is what is supplying the movement information to AR.js. We'd have to replace this or add more functionality to make it marker less. We could add something like OpenCV to do the feature point detection.

TLD;DR Difficult but maybe possible

[1d10t]

hi2all!

i wrote three basic aframe components for gps- and compass-based markerless ar (see code at https://github.com/1d10t/test/blob/16b9a59ec9fcacfaa7b5c485622f21c7f4191fbb/a-frame-webcam-dae.html)

'gps-place' component is for object entity 'gps-position' and 'compass-rotation' components is for camera

in result is not too smooth, cause of compass and gps sensor is laggy. maybe need some algorithmic workaround

[jeromeetienne]

hi @1d10t

i love what you did there. and im finnaly got the time to integrate it :) unfortunatly i can't make your example to work. i run it but nothing appears on the screen beside the webcam... any hint on how to make it work ? :)

What i am doing wrong ?

[jeromeetienne]

@1d10t i know what i was doing wrong, i wasnt updating the longitude/lattitude :)

[1d10t]

there was a bug in calculation of move direction, latest commits of this file is better. also u can test simplified example with sphere primitives https://1d10t.github.io/test/phills-sphere.html

[1d10t]

i expect that we can use some sort of LLA-markers to stabilize position in highly noisy locations, where GPS like a crazy bunny

[jeromeetienne]

@1d10t i would love to talk about how to improve the accuracy

[jeromeetienne]

@1d10t i noticed you dont handle altitude in your code. is that just small shortcut or is there an issue with handling altitude ?

[1d10t]

just didnt try

[raynt]

I am very interested in testing this out. From the example: https://github.com/1d10t/test/blob/16b9a59ec9fcacfaa7b5c485622f21c7f4191fbb/a-frame-webcam-dae.html

What would need to be edited in this file to test? I adjusted the latitude and longitude but don't see any AR object appearing at these coordinates. Would be very thankful for a quick run through how to get this example running.

[AndreaMonzini]

Hello, i just tested it and it works on iPad Air® with iOs® 11 !

@1d10t great work! are your components under MIT license like AR.js?

@raynt 3D .dae object are commented, just delete comments ;)

Testing: http://amvr.it/test/agps.html

[raynt]

Thank you for the reply Andrea. When I delete the comments I just get a white screen.

`       <a-asset-item id="fence_50_31_0040507_9_asset" src="fence_50_31_0040507_9.dae"></a-asset-item>
        <a-asset-item id="fence_50_31_0040507_6_asset" src="fence_50_31_0040507_6.dae"></a-asset-item>
        <a-asset-item id="fence_50_31_0040507_16_asset" src="fence_50_31_0040507_16.dae"></a-asset-item>
        <a-asset-item id="fence_50_31_0040507_25_asset" src="fence_50_31_0040507_25.dae"></a-asset-item>

`

If I set a long/lat coordinate for across the street will I be able to see the 3d object looking across from the opposite side of the street or does the mobile device need to be at that specific long/lat for the 3d object to appear?

[AndreaMonzini]

@raynt i keep testing on Android® and GPS this time and it works ( a bit laggy but ok). Have you changed Longitude and latitude coordinates near to your actual position? The object appears on the gps-place="longitude: xx.xxxxxxx; latitude: xx.xxxxxxx" from the perspective of the mobile device coordinates. Remember to delete comments from <a-entity> too.

[1d10t]

example with reduced code https://1d10t.github.io/test/phills-sphere.html @AndreaMonzini MIT

[AndreaMonzini]

@1d10t Thank you :)

I'm testing it with different 3D Object, trying to move around the position and it's working but the position is quite trembling probably for the jitter of the sensor data.

[raynt]

Thanks @AndreaMonzini and @1d10t I got it working as well with your assistance. It was very unstable but I was testing indoors so maybe the GPS signal is weak. Interesting to see if it works better outside.

[AndreaMonzini]

Hello i found this very interesting project by @andygup , with Apache License Version 2.0, and it could be useful to reduce the jitter:

https://github.com/Esri/html5-geolocation-tool-js

In particular:

https://github.com/Esri/html5-geolocation-tool-js/blob/master/js/GeolocationHelper.js

[andygup]

@AndreaMonzini Thanks for the mention! There are three ways to reduce location fluctuations on the current generation of smartphones: location averaging (GelocationHelper.js is one example), have user turn off all data access to Cellular and Wifi (not always possible or ideal) to force the device to use GPS-only, and/or use a Cordova plugin that separates NETWORK locations from GPS locations.

Good news is that high-accuracy GNSS-enabled phones are coming (hopefully) next year.

[javismiles]

Im super interested in this as well, thank you @1d10t @AndreaMonzini MIT for your great work, question, I tried your example https://1d10t.github.io/test/phills-sphere.html , on android, and I can see the white sphere right on the place where the mobile phone is. Now, if I wanted to put the sphere x meters in front of me, what and where would I have to change the code? I see that at the moment as I move the phone, the sphere follows the position of the phone, I change the gps-place coordinates, but object remains in same place always, how can I station the object so that its always either in the same global coordinates independently of the mobile phone position or better in the same relative position x distance away in relation to the mobile phone position?

Update: I wasn't testing right yesterday, now I have put the right latitude and longitude coordinates and now the testing works well, I can see and move around the entity but of course no precision, it jumps all over the place, but I can see the potential indeed

thanks a lot again ;)

[raynt]

@javismiles this example requires Unity and Kudan SDK ( https://www.youtube.com/watch?v=7-SiR_qJwaU ) but I think it helps explain what you are looking for (scroll to end of video). Basically the 3d model stays in an anchored position and you can walk closer, further away or around the object. This would be an amazing addition to AR.JS

[1d10t]

@javismiles try to go further and you will see, that spheres moves with large jumps by several meters. if you've ever played in ingress (by niantic-google), you might have noticed similar accuracy.

[javismiles]

@raynt, @1d10t thank you for your messages, so @raynt yes that's exactly what I mean, and this is something that Jerome showed recently in his first experiment combining Google ArCore and ar.js https://www.youtube.com/watch?v=I1foLzF8slc , I wonder how these ar.js - arcore experiments tie in with this other markerless gps based approach, in any case I can see that the final effect is similar except that here we have at the moment a lack of precision, the positioned object jumps all over the place because of the gps accuracy issues I guess,

and @1d10t so you mean that if i move around a lot the sphere won't stay in the center but will jump all around, ok so you are right yes, I just made new tests, I wasn't testing correctly yesterday, now I get it all, I input my latitude and longitude and now yes, I can see the box entity I added to the scene and I can move around it but yes it jumps all around and it's not precise at all, I get you all now yes, so it works but it's not precise at all, we can see the huge potential of it, but it has a total lack of precision right now

I wonder if, as some have suggested above, if LLA markers used indoors would help to make this experiment more precise?

btw, speaking of markerless AR and a bit off topic, another type of markerless ar is feature based detection and last week totally by chance i discovered these people: https://github.com/Tastenkunst/brfv4_javascript_examples https://tastenkunst.github.io/brfv4_javascript_examples/ I don't know if you are familiar with them, but it's pretty amazing...

thank u everybody, this is all fascinating stuff, Ive been playing with ar.js and aframe for a while now and really enjoying all of this

[AndreaMonzini]

@andygup thank you for the explanation! @javismiles of course the system works better outdoor and only if you do not need precise position accuracy, i'm implementing a project in large squares for example.

[raynt]

Thanks for sharing those links @javismiles. @jeromeetienne will you be releasing your implementation of ar.js/arcore?

[enjinstudios]

Any plans to roll this feature out in the near future? Happy 2018!

[jucall]

@1d10t I tried your code (https://1d10t.github.io/test/phills-sphere.html) but I can not see anythings. Maybe because I'm in southern hemisphere, with negatives coordinates (-33.0519424,-71.3836586). Did you tested it before with negatives coordinates?. Greetings.

[tomasdev]

anybody has a good plane detection paper?

[florianmaxim]

So it seems like this is not being developed anymore? That's too bad :(

[geekhunger]

I googled an ~hour and this seemed to be the best framework for AR over the web ... but the markers make it pretty odd to use.

[ajmaurya99]

Still my question is Markerless object placement is possible using AR.js I still haven't found any solution to this. Please share if you know on how to achieve this. Or any other library that I can use to achieve the same on web

[kageori]

I really need information about developing markerless AR. Is this function not developed in AR.js?

[aboutqx]

Seems it's not using gps like arkit or arcore,but an algorithm called slam. link: https://www.kudan.eu/kudan-news/b2-evaluate-slam-accuracy/

[mnutsch]

@aboutqx The current iteration of AR.js is best described as a mashup of ARToolkit (marker detection) and AFrame.js (3D scene rendering).

@kageori Standard mobile web browsers don't expose ARKit/ARCore functionality (yet). If you want to experiment with markerless AR through the web right now, you might want to check out Mozilla's Project WebXR Viewer: https://github.com/mozilla-mobile/webxr-ios

[JessCasasDev]

@1d10t I tried your code (https://1d10t.github.io/test/phills-sphere.html) but I can not see anythings. Maybe because I'm in southern hemisphere, with negatives coordinates (-33.0519424,-71.3836586). Did you tested it before with negatives coordinates?. Greetings.

@jucall Did you manage to run the example?, I'm having the same issue :/

[1d10t]

hi ppl now u can test 'gps-place' and 'gps-position' components on another example with your PC

1. open https://1d10t.github.io/test/a-frame-dae-gmap-ground.html

2. run code in console camera.components['gps-position'].zeroCrd = {latitude: -33.0519424, longitude: -71.3836586}; camera.components['gps-position'].crd = {latitude: -33.0519424, longitude: -71.3836586}; camera.setAttribute('position', camera.getAttribute('position')); document.querySelector('a-scene').append(ce('a-sphere', {'gps-place': "longitude: -71.3836586; latitude: -33.0519424"}));

3. sphere will appears under your virtual location

4. use wasd controls to move camera

[jucall]

@1d10t great work !!! Thanks. Let's try it @jlcasasg

[evaristoc]

Hi,

I confirm I saw this example on Samsung Galaxy S7, Android Oreo: https://1d10t.github.io/test/a-frame-dae-gmap-ground.html

This one could see updating parameters but no sphere: https://1d10t.github.io/test/phills-sphere.html. The size of the scene didn't fit the size of screen.

Both tested indoors.

[evaristoc]

Is this part of the project developed somewhere I could watch to see if I could help/test? It seems you are using some ML for image stabilization? I probably could help.

[evaristoc]

@1d10t @jeromeetienne and all:

Have you decided which technologies / solutions you would like to work for the markerless example?

• LLA (something like what @1d10t has been working so far)?
• SLAM?
• other(s)?

[evaristoc]

@1d10t for the many (excellent) examples you made using mobile location sensors, argon.js appears to do more or less what your code does.

If I understand correctly what argon.js tries to do differently is making you the 0,0 coordinate and re-calculating all the coordinates to any point in the world from you as reference. In that way you would not be mapped incorrectly if the precision fails: no matter your GPS coordinates, you are always 0,0.

The problem in both cases resides in the lack of precision and the variability of the GPS data. In your case the positioning is as accurate and precise as the GPS data. In the case of argon.js the positioning is more accurate but still unstable, ranging between 2 and 100mts away from my actual position. If I understand correctly the argon.js approach, I think its problem is the speed at which the GPS data changes versus the time it takes to recalculate the reference points. It is possible that by the time I am setup as 0,0 the GPS data changed so all the relations become wrongly estimated.

I actually don't know.

Here an explanation of how argon.js works: https://docs.argonjs.io/tutorial/part1/

They recommend the following (which now make sense...):

Since the local reference frame may change at any time, a programmer should not save and use the values in this frame for more than a single update and render step. If the values need to be saved and used over multiple frames, it is possible to be notified when the local frame of reference changes.

The argon.js repository is now deprecated: its authors are putting their efforts in what it seems the main focus of several organizations nowadays: XR browsers - I think @jeromeetienne is also involved in one of those initiatives? Despite of being deprecated I would recommend you to check it out! They have been using another library (cesium.js) to estimate positioning.

[droid001]

I'm guessing one needs a tango phone to run the markerless tracking method in ar.js which Jerome was writing about in this Medium article?

[FangerZero]

I found this but can't seem to get any WebAR to work on my Pixel XL regardless of what I try... I can't even get the Google Samples to work... https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e

[jonnyscholes]

RE Google's samples the spec is very unstable at the moment and the samples lag behind what's implemented. My understanding is that they only work on Chrome Canary <71.

More details here: https://bugs.chromium.org/p/chromium/issues/detail?id=918375&q=component%3ABlink%3EWebXR%3EAR%20&colspec=ID%20Pri%20M%20Stars%20ReleaseBlock%20Component%20Status%20Owner%20Summary%20OS%20Modified

On Thu, Feb 7, 2019, 7:50 AM Fanger Zero notifications@github.com wrote:

I found this but can't seem to get any WebAR to work on my Pixel XL regardless of what I try... I can't even get the Google Samples to work...

https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/jeromeetienne/AR.js/issues/190#issuecomment-461182109, or mute the thread https://github.com/notifications/unsubscribe-auth/ABM3ZzfTxjLR_GitHZzpK6vM_V6Qm38Nks5vK0ARgaJpZM4PacoQ .

-- Jonny Scholes Web Developer, Photographer & Designer Takeflight | Web Developer 42° | TotesProfesh

www.jonnyscholes.com

[commentatorboy]

I found this but can't seem to get any WebAR to work on my Pixel XL regardless of what I try... I can't even get the Google Samples to work... https://medium.com/@urish/web-powered-augmented-reality-a-hands-on-tutorial-9e6a882e323e

It is because you have to download the chrome-like browser for it work. https://github.com/google-ar/WebARonARCore.

I'm guessing one needs a tango phone to run the markerless tracking method in ar.js which Jerome was writing about in this Medium article?

Yes you do.

So it seems like it is not possible to do markerless AR in ar.js in common browsers. And just to notify people. Tango is now called ARCore. So I do not know if Tango is supported in any way.

So for anything to work you have to install: https://play.google.com/store/apps/details?id=com.google.ar.core&e=-EnableAppDetailsPageRedesign

Also you can see here how it should be done. https://developers.google.com/web/updates/2018/06/ar-for-the-web

[tomasdev]

For those needing plane detection I highly recommend 8thwall.com