Three.js WebXR AR examples don't recognize our WebXR API

takahirox commented 4 years ago

Three.js WebXR AR examples don't recognize our AR capability. When the application starts up Three.js ARButton checks navigator.xr but at that time it seems navigator.xr isn't defined yet.

Android Chrome (native WebXR API) and WebXR Emulator Extension work fine. So maybe there is something problem in our side.

blairmacintyre commented 4 years ago

are we not definitely navigator.xr in the shim? Didn't something change to navigator.XR or something? (fuzzy memory here)

takahirox commented 4 years ago

I need to investigate but I guess WebXR API detection with navigator.xr is somehow done before loading shim. When is shim exactly loaded?

blairmacintyre commented 4 years ago

That's not possible or shouldn't be. The shim is loaded using the WkWebView feature that preloads some JS before loading the page. Are you sure the shim that's bundled with xcode is actually setting .xr and not something else like .XR?

takahirox commented 4 years ago

Yes, navigator.xr.

I'll investigate closer today.

takahirox commented 4 years ago

I found an interesting fact.

They are the same Three.js AR paint example. Our WebXR API is recognized in 2. while it isn't in 1.

The difference is whether using <iframe>. 2. is the WebXR AP paint example itself and 1. opens 2. in <iframe>.

I set break point in the example in 1 and realized that navigator.xr seems to be never defined in <iframe>.

So my speculation is the shim has no effect to a page loaded in <iframe>. Can this likely happen from the view of preloading with WkWebView?

blairmacintyre commented 4 years ago

Yes, that could definitely be the case.

Is webxr supposed to work in iFrames? A subset of it? Seems dangerous.

joshmarinacci commented 4 years ago

I seem to recall us discussing webxr in iframes about a year ago. I think we agreed to have some property set on the iframe to enable it.

takahirox commented 4 years ago

From Three.js html code

<iframe id="viewer" name="viewer" allowfullscreen allowvr onmousewheel="">

It has allowvr (not "xr" tho), is this what @joshmarinacci pointed out?

takahirox commented 4 years ago

And, I'm not familiar with WebKit at all. Is it technically possible to preload the shim even in <iframe>?

blairmacintyre commented 4 years ago

It has allowvr (not "xr" tho), is this what @joshmarinacci pointed out?

Probably.

And, I'm not familiar with WebKit at all. Is it technically possible to preload the shim even in ?</p> </blockquote> <p>I <em>think</em> so but I can't remember, nor remember how.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/kearwood"><img src="https://avatars.githubusercontent.com/u/3256256?v=4" />kearwood</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <blockquote> <p>I seem to recall us discussing webxr in iframes about a year ago. I think we agreed to have some property set on the iframe to enable it.</p> </blockquote> <p>WebXR uses feature policy to enable or disable it in iframes:</p> <p><a href="https://immersive-web.github.io/webxr/#feature-policy">https://immersive-web.github.io/webxr/#feature-policy</a></p> <p>In particular, “xr-spatial-tracking”.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/takahirox"><img src="https://avatars.githubusercontent.com/u/7637832?v=4" />takahirox</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>Thanks @kearwood. My understanding is if <code>xr-spatial-tracking</code> attribute isn't specified <code>navigator.xr</code> should be defined but <code>isSessionSupported('immersive-vr/ar')</code> should return false and <code>requestSession('immersive-vr/ar')</code> should fail, correct?</p> <p>If it's correct, we need to request firefox-ios to load the shim even in iframe.</p> <p>I guess we can't know iframe attribute from inside of iframe. Then we add another shim for <code>xr-spatial-tracking</code> disabled case and firefox-ios preloads an appropriate one depending on iframe attribute?</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/blairmacintyre"><img src="https://avatars.githubusercontent.com/u/1348861?v=4" />blairmacintyre</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>Let's leave this open for now and investigate</p> <p>On Fri, Mar 27, 2020 at 1:37 PM Takahiro <a href="mailto:notifications@github.com">notifications@github.com</a> wrote:</p> <blockquote> <p>Thanks @kearwood <a href="https://github.com/kearwood">https://github.com/kearwood</a>. My understanding is if xr-spatial-tracking attribute isn't specified navigator.xr should be defined but isSessionSupported('immersive-vr/ar') should return false and requestSession('immersive-vr/ar') should fail, correct?</p> <p>If it's correct, we need to request firefox-ios to load the shim even in iframe.</p> <p>I guess we can't know iframe attribute from inside of iframe. Then we add another shim for xr-spatial-tracking disabled case and firefox-ios preloads an appropriate one depending on iframe attribute?</p> <p>— You are receiving this because you commented. Reply to this email directly, view it on GitHub <a href="https://github.com/MozillaReality/webxr-ios-js/issues/80#issuecomment-605145691">https://github.com/MozillaReality/webxr-ios-js/issues/80#issuecomment-605145691</a>, or unsubscribe <a href="https://github.com/notifications/unsubscribe-auth/AAKJJ7P3L6LL4XBTOKC3X7TRJTP57ANCNFSM4LTC3ZSQ">https://github.com/notifications/unsubscribe-auth/AAKJJ7P3L6LL4XBTOKC3X7TRJTP57ANCNFSM4LTC3ZSQ</a> .</p> </blockquote> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/takahirox"><img src="https://avatars.githubusercontent.com/u/7637832?v=4" />takahirox</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>Let's ask Firefox iOS team about how we should do, and update if needed later.</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/takahirox"><img src="https://avatars.githubusercontent.com/u/7637832?v=4" />takahirox</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>I had a comment from firefox-ios team</p> <blockquote> <p>Look for where your script is injected using WKUserScript constructor. That takes an argument forMainFrameOnly , a boolean, set that to false.</p> </blockquote> <p>I did it and our WebXR API is recognized even in a web page in iframe. But I face another error.</p> <pre><code>ReferenceError: Can't find variable: arkitCallback_initAR_1588968895079_942515975864898 Global Code -- Script Element 3:1</code></pre> <p>I guess the bridge between a web page in iframe and arkit doesn't work. I'm not familiar with the bridge yet but does my guess sound correct? And anyone knows how to resolve?</p> </div> </div> <div class="comment"> <div class="user"> <a rel="noreferrer nofollow" target="_blank" href="https://github.com/blairmacintyre"><img src="https://avatars.githubusercontent.com/u/1348861?v=4" />blairmacintyre</a> commented <strong> 4 years ago</strong> </div> <div class="markdown-body"> <p>Ah, of course. iFrames can't talk to the native app. It can only talk to the parent of the frame (the page), I believe.</p> <p>If we want to enable iFrames we have to do this in the shim:</p> <ul> <li>detect that we're in an iframe</li> <li>instead of calling the native hooks, send messages to the parent frame</li> <li>in our main page shim, look for those messages, and relay them to the native app</li> <li>in our main page shim, look for replies/message from native, and relay them to the iframe</li> </ul> <p>This will be a non-trivial amount of work; straightforward I think, but still not trivial.</p> <p>We'll also need the page to keep track of multiple iFrames (e.g., only one can create an immersive session at a time, and we need to relay the data to the right one).</p> </div> </div> <div class="page-bar-simple"> </div> <div class="footer"> <ul class="body"> <li>© <script> document.write(new Date().getFullYear()) </script> Githubissues.</li> <li>Githubissues is a development platform for aggregating issues.</li> </ul> </div> <script src="https://cdn.jsdelivr.net/npm/jquery@3.5.1/dist/jquery.min.js"></script> <script src="/githubissues/assets/js.js"></script> <script src="/githubissues/assets/markdown.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/highlight.min.js"></script> <script src="https://cdn.jsdelivr.net/gh/highlightjs/cdn-release@11.4.0/build/languages/go.min.js"></script> <script> hljs.highlightAll(); </script> </body> </html>

MozillaReality / webxr-ios-js

Three.js WebXR AR examples don't recognize our WebXR API #80