search

stalgiag / p5.xr

a library that helps turn p5.js sketches into immersive experiences using WebXR
GNU Lesser General Public License v2.1
117 stars 25 forks source link

Refocusing on VR and marker-tracking due to updates in iOS WebXR compatibility #103

Open stalgiag opened 3 years ago

stalgiag commented 3 years ago

After more than a year of waiting for a solution that would allow WebXR on iOS, it seems like this is not worth waiting for anymore. The most promising solution was being developed by Mozilla but it still has a number of significant bugs and isn't fully compatible with the current WebXR api. It was my understanding that another release was on the horizon but with recent layoffs this seems unlikely.

These developments have caused me to reconsider what this project can be useful for. And with this consideration, I think it useful to focus on VR and marker-based AR. These are available cross-platform and do not require waiting for updates for others. In keeping with the mission of p5, I think it is most important to ensure cross-platform compatibility.

I will work on polishing the project for these uses over the next few months. Feel free to add any thoughts here.

lossless commented 3 years ago

I agree with the refocusing. As you noted, Mozilla's WebXR Viewer is unlikely to receive any updates. While there are occasional hints that Apple may get more serious about WebXR support in Safari, there are numerous cases where Safari support for a web standard doesn't make for smooth sailing (WebRTC being one). So even if some type of WebXR support arrived on Safari tomorrow, I think there's no guarantee that substantial work - or workarounds - wouldn't be required on your part to make p5.xr work on Safari.

Thank you for continuing work on p5.xr, by the way. I made good use of it on a recent sketch, together with tensorflow.js. https://twitter.com/lossless/status/1307024119892979712 https://attomoto.glitch.me

stalgiag commented 3 years ago

@lossless Thanks so much for sharing this! I never know whether anyone actually uses this because I have been waiting so long to 'release' it.

I agree that waiting on Safari isn't worth it. Apple seems more interested in keeping AR in a native app space where they have control over the 'quality of the experience'. This is a bummer because the original goal with this project was to make something that educators working with p5 could use in their classroom to open XR up to their students with minimal new knowledge required. But I think teachers, at least in the US, are not going to work with something that won't support most (or all) of their students' phones.

Anyways -- thanks for sharing! It is great to know that someone has gotten some use out of this!

TiborUdvari commented 2 days ago

Just an update on the situation: considering the use case for mixed reality headsets like the Meta Quest 2/3 and the Vision Pro, the specs are unlikely to give direct access to the camera feed because of privacy concerns.

Marker-based tracking should be implemented when the spec will be supported https://github.com/immersive-web/marker-tracking

If we choose a solution that uses the camera feed directly, as can be accessed with smartphones, it will diverge too much from the functionality of other devices that support the spec, making it hard to maintain.