WebGPU support

[wcandillon]

Starting with r167, WebGPU support in three-fiber broke. @aarongrider created a reproducible example here. We thought we would open an issue to learn more about this topic and track it.

Interestingly enough, @aarongrider and I are not particularly interested in WebGPU support in three-fiber. What we really want is a truly portable reconciler API that we can use in any context. We are writing our own Canvas implementation and ran into runtime issues while doing so. This led us to try WebGPU support in three-fiber.

Our goal is to run three-fiber on React Native with WebGPU. To achieve this, we need to write our own Canvas element (and use the web version of three-fiber, not the native one). Ultimately, we only need the reconciler. We plan to run the reconciler in its own worklet thread, use Reanimated for animations (e.g., no useFrame), use gesture-handler for gestures, handle asset loading outside of three-fiber, and more. We had built a tiny working example, but encountered several runtime issues with more advanced examples.

[CodyJasonBennett]

You've probably been impacted by https://github.com/mrdoob/three.js/issues/29156 as that breaks compatibility with everything unless you change bundler resolution for three. This is an upstream issue which must be resolved, but no further action is needed than using an environment which supports ESM and top-level-await, and constructing the renderer with <Canvas gl={(canvas) => new WebGPURenderer({ canvas })} />. Some further context: https://x.com/Cody_J_Bennett/status/1821555844980285515.

[wcandillon]

we are able to run three.js r168 out of the box, it‘s with three-fiber that we hit issues. You mentionned in another issue that the rexonxiler doesn‘t do anything tied to particular three.js API therefore we were surprised to experience these differences of behavior.

On Fri 13 Sep 2024 at 17:19, Cody Bennett @.***> wrote:

You've probably been impacted by mrdoob/three.js#29156 https://github.com/mrdoob/three.js/issues/29156 as that breaks compatibility with everything unless you change bundler resolution for three. This is an upstream issue which must be resolved, but no further action is needed than using an environment which supports ESM and top-level-await, and constructing the renderer with <Canvas gl={(canvas) => new WebGPURenderer({ canvas })} />. Some further context: https://x.com/Cody_J_Bennett/status/1821555844980285515.

— Reply to this email directly, view it on GitHub https://github.com/pmndrs/react-three-fiber/issues/3352#issuecomment-2349204844 or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKXVWJUIAIOODTBCORXPTZWL7ABBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVEYTOMRVGIYTSMRWQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSGUZDIMBQGUYDINVHORZGSZ3HMVZKMY3SMVQXIZI . You are receiving this email because you authored the thread.

Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[CodyJasonBennett]

There's nothing mystical or different about how three behaves when it interacts with R3F, so I would strongly suggest you start with vanilla three and get that working, and don't change that when you introduce the R3F canvas. I don't have the time to debug your app, but there's definitely something wrong or different if you're seeing any runtime difference at all, with https://github.com/mrdoob/three.js/issues/29156 as a key culprit.

[wcandillon]

this is the approach we took but we may have done some mistakes, we will try again more rigorously and report back. This definitely answersing our question so far, thank you.

On Fri 13 Sep 2024 at 17:31, Cody Bennett @.***> wrote:

There's nothing mystical or different about how three behaves when it interacts with R3F, so I would strongly suggest you start with vanilla three and get that working, and don't change that when you introduce the R3F canvas. I don't have the time to debug your app, but there's definitely something wrong or different if you're seeing any runtime difference at all, with mrdoob/three.js#29156 https://github.com/mrdoob/three.js/issues/29156 as a key culprit.

— Reply to this email directly, view it on GitHub https://github.com/pmndrs/react-three-fiber/issues/3352#issuecomment-2349231585 or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKXVS3VDBINPVZMQXPQ7TZWMAO5BFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVEYTOMRVGIYTSMRWQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSGUZDIMBQGUYDINVHORZGSZ3HMVZKMY3SMVQXIZI . You are receiving this email because you authored the thread.

Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[wcandillon]

completed in #3353