search

pmndrs / xr

🤳 VR/AR for react-three-fiber
https://pmndrs.github.io/xr/docs/
Other
2.14k stars 155 forks source link

Possibility of Video or I-Frame embed in React XR #214

Closed iamkkt closed 2 months ago

iamkkt commented 1 year ago

As per my understanding, Drei Html component wont render in react XR and Drei Image component wont support video. Is there is any work around for I- Frame or Video embeed in react XR.

CodyJasonBennett commented 1 year ago

I'd look into projecting a VideoTexture onto a plane or adding support for that in https://github.com/pmndrs/drei/blob/master/src/core/Image.tsx if it doesn't already. It should be fine as long as it doesn't rely on window.requestAnimationFrame -- this does not with in an immersive session.

From a quick glance, you should be able to pass any sort of texture and it work without issue.

iamkkt commented 1 year ago

Thank you for replies. I have tried with video texture and works fine for video. As the size of the video increases, it might create a problem for as it cant stream. Is there any work around for this, where i can able to stream like youtube or any possibility of YouTube embed in react XR like drei html ?

iamkkt commented 1 year ago

My requirement is to stream either a video of length 30 min or a YouTube video.

rgsaura commented 1 year ago

Hello! Exited to explore this field. What about inserting an Iframe, any idea?

rekliner commented 1 year ago

This issue is not related to react-XR. https://github.com/pmndrs/drei#usevideotexture is your answer for displaying video from a file or stream on a surface.

@iamkkt To get video content from a service like youtube or vimeo means reverse engineering their player, something they explicitly make difficult. By design you can't directly access their video content, and you will not find an out of box solution. Any method that is popularized is quickly obfuscated in response by those companies and their methods are proprietary. Not to say its impossible but it will likely always be a very technical hurdle and not the realm of a simple material import.

Regardless, it would happen outside of the realm of react-xr, drei, and three.js. Nothing can be done here to help you, but I don't mean that to be unhelpful. There are other ways to host and stream video and you can certainly show a video stream on a surface. Start learning with the basic R3F video texture example here: https://codesandbox.io/s/39hg8 ... Lean about video streams with this sandbox: https://codesandbox.io/s/threejs-video-cube-nmjch8

Once you have the video stream available useVideoTexture() is your resource in the 3d scene. By here I mean drei, not react-xr. Your second step is to attach that video to an object, likely a plane but it can be anything, as demonstrated in the examples above.

netgfx commented 1 year ago

On a similar note a simple mp4 video is playing fine, until entering the AR mode a second time then the video stops, and any subsequent try to enter AR the video stops immediately. Is there a limitation or something that needs to be enabled in order for the video to work every time someone enters AR mode?

bbohlender commented 2 months ago

Hello! Exited to explore this field. What about inserting an Iframe, any idea?

Iframes do not work. Check out uikit for this use case.

until entering the AR mode a second time then the video stops, and any subsequent try to enter AR the video stops immediately

@netgfx Can you check if that behavior persists with react-three/xr v6 and the latest Quest Browser (or whatever browser + device you are using) since I have not yet experienced this behavior. If so, please open another issue related to that :)