WebXR support? - Githubissues

npolys commented 3 years ago

It would be amazing to have support for

WebXR https://www.w3.org/TR/webxr/

example WebVR migration process documented here: https://github.com/immersive-web/webxr/blob/master/webvr-migration.md

Groogey commented 1 year ago

I was wondering if there has been any progress in this field as x3dom just added native webxr support to their browser

create3000 commented 1 year ago

I haven't done anything special in this direction yet, but it is of course always possible to control the X3D nodes via JavaScript, and thus achieve the desired result.

https://create3000.github.io/x_ite/reference/script-node-authoring-interface

create3000 commented 2 months ago

With the latest release (10.3.0) there is now basic WebXR support built directly into X_ITE. If WebXR is possible then a new button (Cardboard Glasses) appears in the lower right corner of the \<x3d-canvas>. In addition, there are now three new attributes (xrButton, xrMovementControl, xrSessionMode) and the corresponding browser options (XRButton, XRMovementControl, XRSessionMode).

If anyone has a VR device and can test this new feature, I would be very happy to receive feedback.

Sgeo commented 2 months ago

Tried in in Meta Quest 3: Native Browser, Wolvic, and Oculus Air Link -> Chrome (with Oculus set as OpenXR runtime). Tested on the X_ITE homepage.

The right eye looked ok, although maybe some lighting oddities (on the Wikipedia logo sample). Although I'm not entirely sure.

The left eye did not work well. The image was weirdly distorted and moved in weird ways.

create3000 commented 2 months ago

@Sgeo I was able to identify a possible source of error. I don't want to bother you any further here, but if you like, please test again on the page with the link below, and give some feedback:

https://rawgit.com/create3000/x_ite/xr/x_ite.min.html

npolys commented 2 months ago

NB: you can do a lot of testing by installing a WebXR emulator for your browser like chrome or ffox...

then you can launch a session and see the stereo view, and even control the head and controllers through the developer console!!

https://addons.mozilla.org/en-US/firefox/addon/webxr-api-emulator/ [https://addons.mozilla.org/user-media/previews/full/228/228420.png?modified=1622134679]https://addons.mozilla.org/en-US/firefox/addon/webxr-api-emulator/ WebXR API Emulator – Get this Extension for 🦊 Firefox (en-US)https://addons.mozilla.org/en-US/firefox/addon/webxr-api-emulator/ Download WebXR API Emulator for Firefox. Emulate WebXR devices on your browser addons.mozilla.org

From: Holger Seelig @.> Sent: Wednesday, August 21, 2024 5:45 AM To: create3000/x_ite @.> Cc: Polys, Nicholas @.>; Author @.> Subject: Re: [create3000/x_ite] WebXR support? (#75)

@Sgeohttps://github.com/Sgeo I was able to identify a possible source of error. I don't want to bother you any further here, but if you like, please test again on the page with the link below, and give some feedback:

https://rawgit.com/create3000/x_ite/xr/x_ite.min.html

— Reply to this email directly, view it on GitHubhttps://github.com/create3000/x_ite/issues/75#issuecomment-2301623351, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB2TSM33P6MDRENFJ7XNWMDZSROTHAVCNFSM6AAAAABMWNTMJSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBRGYZDGMZVGE. You are receiving this because you authored the thread.Message ID: @.***>

create3000 commented 2 months ago

Thanks @npolys, let's put it this way: with the emulator I was already through with testing and have cleared all obstacles, but as it turns out now a real life device is a bit more peculiar.

It's already on my wish list to buy a VR headset myself, but by then the biggest pitfalls can be removed.

npolys commented 2 months ago

Cool~!

I will get my students in the lab to test it!

stay tuned! 🙂

br, _n

From: Holger Seelig @.> Sent: Wednesday, August 21, 2024 10:08 AM To: create3000/x_ite @.> Cc: Polys, Nicholas @.>; Mention @.> Subject: Re: [create3000/x_ite] WebXR support? (#75)

Thanks @npolyshttps://github.com/npolys, let's put it this way: with the emulator I was already through with testing and have cleared all obstacles, but as it turns out now a real life device is a bit more peculiar.

It's already on my wish list to buy a VR headset myself, but by then the biggest pitfalls can be removed.

— Reply to this email directly, view it on GitHubhttps://github.com/create3000/x_ite/issues/75#issuecomment-2302146324, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB2TSM3DAKL7WQ3STQL2PMDZSSNNFAVCNFSM6AAAAABMWNTMJSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMBSGE2DMMZSGQ. You are receiving this because you were mentioned.Message ID: @.***>

Sgeo commented 2 months ago

I'm happy to help!

Left eye is now fixed.

There's a weird issue where the scene doesn't seem to load the first time I press the XR button. Going to my Oculus/Meta menu and back causes the scene to appear.

Backgrounds do not work properly. They rotate in a weird way. Basically any world with a background is a problem.

There is something weird with lighting that it shifts as I rotate my head.

Are there plans for supporting moving around with thumbstick or similar?

create3000 commented 2 months ago

@Sgeo I think we can have it all, at least that's the plan.

Sgeo commented 2 months ago

I believe I found the cause of the issue with the scene not loading: When creating the session, you call addBrowserEvent(), which will then use the session's requestAnimationFrame, but addBrowserEvent returns early if it thinks the scene is dirty (I'm a bit confused by that tbh). Making a variant of addBrowserEvent that unconditionally runs its code seems to work for me, although I'm not sure of the cleanest approach code-wise. Incidentally, I only reproduced the original behavior with moving scenes (like the dolphin one).

Sgeo commented 2 months ago

Are all objects in the scene, including backgrounds, affected by the WebXR projection matrix? If not, that would be my guess as to the backgrounds issue. It looks to me as though projection matrices are generated in a few places, and many of them won't be the WebXR matrices.

create3000 commented 2 months ago

Thanks for tracking down the addBrowserEvent issue. I have made some changes in the 'xr' branch. The projection matrix of the background is now from the XR device and I tried to solve the addBrowserEvent issue, I hope it worked.

The changes are online in the 'xr' branch test page: https://rawgit.com/create3000/x_ite/xr/x_ite.min.html

Sgeo commented 2 months ago

Entering VR now works smoothly 😄

Backgrounds look good, especially in the dolphin world.

Lighting still seems to shift around as I look around, very notable on the Wikipedia logo.

The QuadSet world with X_ITE on it did not work well.

The ShaderPart world with the beautiful torus did not work well, it stayed in place as I turned my head and had double vision, as though neither matrix was being applied.

Background (with rain), and TextureBackground (with snow), were both beautiful in VR. For the snow, are the snowflakes billboards? They seemed to stay facing me and stay in whatever direction I turned my head, but I don't think there's a clean answer for that to make it look less... strange.

I don't remember which worlds, but there was one where I was animated with the viewpoint (which I think is preferable), and one where that didn't happen.

For one of the worlds with shadows, I think that had a viewpoint called Orbiter, the shadows seemed to be in different positions per eye

create3000 commented 2 months ago

This is good news.

Unfortunately, all the worlds were created without a VR headset in mind and on the head. So it's only natural that some work better with a VR headset and some worse. But if you can test a world in parallel with a VR headset while creating it, I think you can come up with some very nice things.

The ShaderPart world is a rectangle with the size of the screen, with a ray tracing shader. So no 3D effect with headset, because left and right eye have the same image.

Yes snowflakes are sprites, that's how ParticleSystem works, if you configure it that way.

Only the active layer, i.e. where navigation takes place, will have the viewer pose position and orientation. This can be set in LayerSet.activeLayer. All other X3DLayerNode nodes will have the view of the bound X3DViewpointNode node, and thus the projection matrix of the viewpoint, maybe that's still a problem.

Shadow world with 'Orbiter' viewpoint is NYC. This world looks like the viewpoint is moving, but its only an illusion, because the viewpoint is at rest, and the whole world is rotating. I can confirm here, that the shadows are in different positions per eye. I'll think about how to fix that.

vralexllobet commented 2 months ago

I don't have a headset myself so I just posted in a couple of Oculus FB groups asking if they could test out the demo site here and also if my site works with a headset. In case you're interested, my site is... https://vr.alexllobet.com/ Thanks

create3000 commented 2 months ago

@vralexllobet I just released a new version v10.4.1 with the latest fixes. You should at least update to this version to get WebXR working.

<script src="https://cdn.jsdelivr.net/npm/x_ite@10.4.1/dist/x_ite.min.js"></script>

Sgeo commented 2 months ago

Is there a way that animated viewpoints can be made to work? Although that may be uncomfortable for some users, I think some users might enjoy that.

What I don't entirely understand is why static viewpoints work and animated ones don't. I would have thought, if only the headset's view positions are used, that that would be relative to 0,0,0. But I can change viewpoints, reenter VR, and see the world from the new viewpoint. Unless it's an animated one.

Sgeo commented 2 months ago

On the very right edge of my vision in my right eye, things sometimes pop out of existence. I'm assuming some culling calculation has gone wrong. Somewhat surprised that it isn't at both edges.

Sgeo commented 2 months ago

Leaving WebXR via the browser's quit option sometimes sends me to weird locations. Clicking the VR button does not do anything.

create3000 commented 2 months ago

It is already possible to animate a viewpoint, but the viewpoint must be in a layer that is not the active one.

LayerSet {
   activeLayer -1
   layers [
      Layer {
         Viewpoint { }
# Animate viewpoint here.
      }
   ]
}

The active layer is the layer where navigation takes place, and this is with WebXR the viewer pose from the headset.

Another way would be, maybe better and easier, to have a browser option/attribute XRMovementControl with values VIEWER_POSE, VIEWPOINT. If it is VIEWPOINT then projection matrix is from pose, view matrix (position, orientation) from viewpoint.

vralexllobet commented 2 months ago

Someone in a FB group made a short video of entering my site. https://youtu.be/qbbCpxmJkfc Seems that the earth image and the stars are not visible. Maybe someday I'll get a headset and fix some things. They also say that they see the buttons on the demo site.

create3000 commented 2 months ago

@vralexllobet very cool. 😎

vralexllobet commented 2 months ago

@vralexllobet very cool. 😎

Thank you! Someone in the FB VR group has tested my site quite a bit. The issue they have is they cannot navigate my worlds except by walking great distances. The person tells me that it would be great to have a link to the buttons on their hand controllers for moving forward and around, touching, grabbing, etc. Is there a page with code examples that I could put into my web page and/or x3d code to connect my worlds to their hand controllers?

npolys commented 2 months ago

YES

we can confirm

HTC vive seemed to have software issues launching WebXR so we will have to revisit.

but Meta Quest 2 and Pro are rendering the X_ITE Treasure island very nicely~!:-)

Yes folks expect to be able to travel using the controller... the original X3DOM WebVR used a head-based navigation.

In the WebXR version, we want allow to point and fly (implemented in X3DOM 1.8.3)

see: https://dl.acm.org/doi/10.1145/3611314.3615918

It is an open question how we can leverage the built-in X3D navigationinfo system since we are ignoring the type='' currently

br, _n

From: vralexllobet @.> Sent: Friday, August 30, 2024 9:06 AM To: create3000/x_ite @.> Cc: Polys, Nicholas @.>; Mention @.> Subject: Re: [create3000/x_ite] WebXR support? (#75)

@vralexllobethttps://github.com/vralexllobet very cool. 😎

Thank you! Someone in the FB VR group has tested my site quite a bit. The issue they have is they cannot navigate my worlds except by walking great distances. The person tells me that it would be great to have a link to the buttons on their hand controllers for moving forward and around, touching, grabbing, etc. Is there a page with code examples that I could put into my web page and/or x3d code to connect my worlds to their hand controllers?

— Reply to this email directly, view it on GitHubhttps://github.com/create3000/x_ite/issues/75#issuecomment-2321196242, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AB2TSM7NSZJKDUN5PVSI55LZUBU5BAVCNFSM6AAAAABMWNTMJSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRRGE4TMMRUGI. You are receiving this because you were mentioned.Message ID: @.***>

create3000 / x_ite

WebXR support? #75