Render Rive to a WebGL texture

[clementroche]

Hi, first of all thank you for this great lib. I'm not here to describe an issue but to ask for a feature.

I'm trying to implement Rive with three.js (an abstraction of webgl), basically I want to use Rive as a WebGL texture. In my current implementation Rive-webgl renders to a canvas then it's converted as a texture using CanvasTexture ( three.js texSubImage2D abstraction), unfortunately this process takes a lot of memory. So i'm wondering if there is a way to render Rive on a WebGL Texture by passing a webgl context instead of a canvas.

I think this can be a game changer in the webgl community since we usually use image spritesheets. It can be a brand new way to import assets in webgl because .riv are lighter than pngs (a lot), it's faster to render and it's easier for collaboration. It's kinda the same vibe as your canvas spritesheet example but for WebGL

Thank you.

[zplata]

Hi! This should be a reasonable request - adding @csmartdalton as he might have more to say on this.

Are you planning to scale your texture?

[clementroche]

Thank you!

Hmmm for the scale question i'm not sure what you have in mind but ideally we should be able to do anything with this texture , this should be a WebGLRenderTarget i guess

[csmartdalton]

Hi @clementroche !

I actually would have recommended rendering to a separate canvas with Rive, and then calling texSubImage2D(..., HTMLCanvasElement) in your other GL context (which appears to be what CanvasTexture does). Sharing a GL context between different libraries that aren't aware of each other gets tricky.

Where are you seeing that this approach takes a lot of memory? The current Rive WebGL runtime is built on Skia CanvasKit, which uses a lot of memory, but I would have expected the browser to implement fast image sharing between contexts for the CanvasTexture operation.

Are you rendering to the canvas using Rive's canvas2d or webgl implementation?

[clementroche]

I'm rendering Rive using @rive-app/webgl. Rive is not the issue actually, it works very well. The memory issue come when calling texSubImage2D using three.js, especially when canvas is huge (>512px).

Here is an example using Lottie, currently rive loader should work the same: draw on a canvas then draw the canvas on a webgl texture. It's causing the same memory issue caused by texSubImage2D

[csmartdalton]

Cool example!

What exactly is the memory issue? Is this Lottie example also calling texSubImage2D with an HTMLCanvasElement?

[csmartdalton]

@rive-app/webgl is very large in size and uses a lot of memory. What if you try using @rive-app/canvas? You should still be able to call texSubImage2D with that canvas, though it might hit a slow path in some browsers when transferring pixels from a canvas2d context to WebGL.

[clementroche]

I have nothing to say about rive renderer, both canvas and webgl are 1000x better than lottie. The memory leak comes from texSubImage2D i could use any canvas even an empty one, rive is not even important in that case, the memory usage increase exponentially with it size. That's a known issue, texSubImage2D is not performant at all on any browser, that's why i'm trying to bypass it

[csmartdalton]

I see, I think I understand the request now. Sorry for the back and forth 😃

So we need the ability to render directly to a texture. E.g.:

gl.bindFramebuffer(gl.FRAMEBUFFER, myRiveFramebuffer);
gl.framebufferTexture2D(gl.FRAMEBUFFER, ..., myRiveTexture, ...);
drawRiveStuff(gl);

As opposed to:

drawRiveStuffToCanvas(myRiveCanvas);
gl.texSubImage2D(..., myRiveCanvas);

This sounds like a good request. The challenge of course will be sharing GL state between the Rive renderer and the client renderer. I see a proposal about shared resources here that could be helpful: https://www.khronos.org/webgl/wiki/SharedResouces

I'll look into this. It seems we might just need some sort of way for each renderer to restore GL state back to initial values in order for the sharing to work.

[clementroche]

Exact, that would be awesome !

[NikolaiArsenov]

Hey team (cc: @csmartdalton)

I hit the same issue in my pet project, where it would be awesome to pass the shared context between my renderer and yours; or render rive into texture to be consumed by my renderer.

Are there any plans to add such feature in the near future? Thank you 🙏