Scope: who can render to our canvas?

[almarklein]

Intro

Our canvas.get_context() always returns a WgpuCanvasContext. I think we can release this a bit, opening options for other tools that want to render to a canvas, without adding much complexity on our end. I did some thinking and experimenting today. In this issue I try to stick to the high-level ideas; I have a PR underway with implementation.

Rendering vs presenting

Let's get some terms straight. The context helps the user-code to render to a canvas. The context presents the result to the canvas. From the p.o.v. of RenderCanvas, we don't care what API the context provides the user with. We only care that the context presents the result in a way that the canvas supports.

            ┌─────────┐      ┌────────┐ 
            │         │      │        │ 
            │ Context │      │ Canvas │ 
            │         │      │        │ 
user        └─────────┘      └────────┘ 
    ───────►          ──────►           
     render            present          

Some thoughts / observations / statements

• I think rendercanvas should be mostly agnostic about wgpu.
• The get_context() in WebGPU is because get_context() already exists for webgl, 2D api, etc. Strictly speaking I don't mind using a completely different API here, but in the end I think this mechanic/abstraction actually works well for our case.
• In the case of wgpu, the context implementation is provided by the wgpu library. That's where the logic is that can turn a window id into a surface id, and configure it. In other words rendercanvas cannot provide the context itself, but must load it from another library.
• I don't want to make an effort to support OpenGL.
• Canvases generally support either rendering to screen, or rendering to an image, sometimes both. Some GUI libs support OpenGL, but see the previous point

Proposal

To get a canvas:

• Use canvas.get_contex(context_type) to allow selecting a context:
  • E.g. canvas.get_contex("wgpu") or canvas.get_contex("something_else")
• Define a hook so that context_type can be "some.module.name", and if that module contains context_from_rendercanvas_hook(), we call it to get the context.
• We can easily implement canvas.get_contex("bitmap") that mimics this. This offers a really simple way for people to e.g. make small games using a numpy array. Simply fun! Also allows creating working examples that don't require tons of lines for wgpu ;)

The contract between the context and the canvas:

• Each canvas backend must support at least either the "screen" or "bitmap" present methods. It may provide more (specialized) methods.
• When a context is instantiated, it gets a present_capabilities, a dict with info on what methods of presentation the canvas supports.
• Each context must support both "screen" and "bitmap" present methods.
  • If a context can render to screen, it's probably a GPU lib, and the texture can be downloaded to cpu.
  • If a context is inherently image-based, it can use a little adapter, that we provide, that presents the image to screen using wgpu.
• The context must implement a method called present(), that returns a little dict that specifies what present method was used, and possibly the data corresponding to the present method.

Future:

• One use-case for more additional, more specialized, render methods is for remote rendering (e.g. jupyter_rfb). E.g. a present method that sends compressed images or diffs.

[almarklein]

BTW: a context being able to present to the screen means that it can make the result appear on screen based only on the window id.

Re opengl: I think its possible to create an OpenGL context that way, So if someone was to create an OpenGLContext that follows our rules, people can use that, and we don't have to care.

[almarklein]

Reading this the next morning. It's rather long, so let's try to summarize it:

Rendercanvas looks independent on wgpu, but right now you can only really render to it with wgpu. I propose to enable other contexts to present to it as well, by formalizing (and tweaking) the API between the canvas and the context. Contexts must support the render-methods that we already have, so the changes are quite minimal. The effect is that other rendering tools can use rendercanvas as well.