ARKit 1.5 new features

[aferriss]

Looks like arkit is getting some nice updates! Supposedly it can now do image tracking (a la vuforia) as well as vertical / wall detection. Seems that they are also up-ing the allowed camera resolution as well as unlocking an autofocus feature (though I can't find docs for this...).

Just opening this issue here as a todo / place for discussion about implementing these features. It's still just in developer beta for now, but supposedly be coming with iOS 11.3 in the Spring.

[sortofsleepy]

🎉

Nice! Thanks for starting this. I was thinking about this too the past couple of days. I haven't had the opportunity to look more in depth but I think there are some things that I think folks could get started on if they wanted to that could be easy enough to implement.

Vertical Plane tracking It looks like it's a simple configuration change - though it's unclear at the moment if you can do both horizontal and vertical tracking at the same time :/

Image Tracking It might be too early to do too much work on this, there is one part though that could be probably be worked on. It sounds like an asset catalog needs to be generated somehow - the docs talk about entering things in XCode but I imagine some kind of build script could get written too, not sure how that would work yet though.

Video Format adjustment If I'm looking at this right - it seems like it's simply another configuration option, though getting the other possible options seems a bit convoluted, I think you have to read out the ARConfiguration.supportedVideoFormats array and set the ARConfiguration.VideoFormat part of the configuration object. Not sure about autofocus yet.

• This however will likely mean changes (or maybe a full re-write) of ARCam as it's currently assuming a camera resolution of 1280x720

My thoughts for now - will edit as needed.

[aferriss]

Here's the sample code page for the image marker stuff for future reference

[sortofsleepy]

Made an 11.3 branch to start tracking new changes for ios 11.3 - I somehow never considered beta releases into how things are structured. All 11.3 related changes ought to go there.

I'll do my best to make sure things are kept up-to-date with develop; though I ran into a bit of a hiccup when trying to test @robotconscience 's updates 😞

[sortofsleepy]

Whew I know it's been awhile since replying to this thread ha.

I thought it might be a good time to try out the "Projects" aspect of Github. To anyone that might wanna submit a PR towards this, I'll be using it to show what I'm doing towards getting this compatible with 11.3. (fellow collaborators - you don't have to use this if you don't want to but I think it'd be helpful to help prevent an overlap of work)

Right now I've taken some time the past two weekends to fix up the ARCam class drop the 1280x720 frame resolution that's currently hardcoded. Turns out it's not actually a big change haha - so also taking the time to clean up the rest of that code as well as try and prep for what I'm sure is the eventual change to the GLM library as I saw in the newer oF nightly versions a couple months back.

[ofZach]

I think you can just use of objects (ofPoint, ofRectangle, ofMatrix4x4) for now and avoid using glm directly -- that way you can avoid having to support both 0.9.8 and the next release as we transition, unless there's specific features of things that glm provides, etc.

[sortofsleepy]

I wasn't terribly certain what the plan was for that part of the transition, something I really also need to look more into and perhaps ask about on the forums. Right now, the changes are really to keep the nomenclature consistent with glm; essentially nothing has really changed, I just added typedefs mapped to existing math classes and swapped out variable declarations.

edit whoops! just found this http://openframeworks.cc/learning/02_graphics/how_to_use_glm/

you wouldn't happen to know how accurate that is now by chance @ofZach? it sounds like the plan is to get rid of the old classes in favor of using glm directly. I think things are fine for now with typedefs but the math stuff might have to change a bit down the line given the more functional approach glm favors.

[sortofsleepy]

Ah ! Also I forgot to mention - I thought it might be useful to have a script based way to quickly pre-process a large quantity of images for image recognition and automatically add them to an assets catalog.

I need to test the theory a little more but I think it can be done since all the files that the normal asset addition process generates are just json but I haven't tried anything out yet.

[cwervo]

+1 @sortasleepy to having a script that can manage loading assets to the asset catalog. This isn't something I have any experience with, but just voicing my support.

[sortofsleepy]

Ok! I know this has basically died out sorta and that's really on me. Things have been a bit hectic to say the least! 😫

I'm stuck in NY for another month finishing up a project - in the meantime my plan over the next couple weeks is to

~* merge Brett's changes in the 11.3 branch into develop. There is a bit of risk as I still can't test (ugh) but no one has thus far complained so I'm assuming things are generally ok. I will make a backup of how develop currently looks before doing so.~ (this has been done - please see the branch pre-11.3 or a previous release if something is wrong)

• start to fix things up to use glm properly as the openFrameworks community has been releasing RCs with glm being used. This is fortunately something I can sort of test as I've used Cinder a bunch and glm has been in there for ages.
• I refactored the camera a bunch but that still needs to be properly tested - gonna hold off on that.

[sortofsleepy]

Ok - so Bret's changes should now finally be in master, it's nice to finally be home and have a working mac haha. It turns out I messed up the merge at first so it took some time to figure out how I messed up and what needed to be re-built but I think I got all of the big bits of code back in. I'm going to keep the 11.3 branch around just in case.

Note that one example that is in the 11.3 branch has been removed - thinking about it more now and being able to properly look at things, it seems a bit redundant to have; if anyone thinks otherwise please feel free to re-add.

Going to close this for now - if there are other 11.3+ additions that anyone would like to see, please add a new issue