Open chirsz-ever opened 2 months ago
We will not add a dependency on winit or any other windowing system, the way we have it currently is the best compromise.
Regarding Canvas and related:
for OffscreenCanvas
, I have a local branch that needs some further work. The idea is that UnsafeWindowSurface
will extend OffscreenCavas
.
We will not implement a webgl context or any webgl related APIs, however we likely will implemented 2dcontext via Vello.
Why don't allow "none" colorSpaceConversion
when createImageBitmap
?
Is this a bug? I think the alpha value do not need to multiply 255. And why there need a is_not_premultiplied
flag calculated from every pixels?
For ImageBitmap
:
ImageBitmap
is designed for immutable bitmap data stored in GPU (but not necessary, for example, Firefox do not store the data in GPU)^1, so it has no interface to get the underlying pixel data.ImageBitmap
:
image::DynamicImage
. This would save memory usage.GPUTexture
as the underlying data resource when possible. This improves performance because you don't need to transfer data from and to the GPU.one you missed that uses ImageBitmap
is https://gpuweb.github.io/gpuweb/#gpuimagecopyexternalimage
- Use pixel format other than RGBA, such as that in
image::DynamicImage
. This would save memory usage.
sounds reasonable
- Use a
GPUTexture
as the underlying data resource when possible. This improves performance because you don't need to transfer data from and to the GPU.
I don't think this is viable, since to create a GPUTexture
you need a GPUDevice
, which is not feasible since a user could use a different GPUDevice
, or same GPUDevice
but with different features.
We could hold the GPUDevice
object in a ImageBitmap
object when the underlying resource is a GPUTexture
. When we want to perform some operations, we first check whether the devices on both sides are the same, and then directly perform the operation, or copy the data to the CPU side and then transfer it to the target.
This optimization would not be implemented in the near future, so it is recorded here just for reference.
So far we have WebGPU and BYOW, but it need more work to use the front-end graphics ecology.
Repository for testing and integrating: https://github.com/chirsz-ever/deno-webgpu-window-demos
Seamlessly use WebGPU-backended frameworks
Bugs and requirements
23518
"image/jpeg"
Creating and managing native windows
We already have deno_sdl2 and dwm for creating windows. But they rely on third-party libraries SDL2 and GLFW.
Maybe we need some methods to create windows without any third-party libraries? I have two ideas:
The first idea is easier to implement, but add a built-in component increases deno's compatibility costs and technical debt.
Canvas
,CanvasRenderingContext2D
andWebGL
Related issues: #5701 #1629
If deno support
Canvas
,CanvasRenderingContext2D
and WebGL, it would be more easy to use front-end graphics libraries.Deno.UnsafeWindowSurface
is analog toCanvas
, and it can usegetContext("webgpu")
to create aGPUCanvasContext
. It is worthwhile to support other contexts,getContext("2d")
,getContext("webgl")
,getContext("webgl2")
andgetContext("bitmaprenderer")
.How to implement CanvasRenderingContext2D (
Canvas.getContext("2d")
)? There are many methods in #5701, which mostly introduces skia or similar graphics libraries like Vello.~For WebGL, we can just use the native OpenGL implementation (gluton)[https://github.com/deno-windowing/gluten], or import ANGLE to maximize compliance of the WebGL standard.~[^1]
Maybe we can do something named "Canvas2D over WebGPU" and "WebGL over WebGPU", which only use JavaScript to implement
CanvasRenderingContext2D
and WebGL with WebGPU.If we resolve all above, implementing
OffscreenCanvas
andImageBitmapRenderingContext
(Canvas.getContext("bitmaprenderer")
) seems not hard.[^1]: Removed by https://github.com/denoland/deno/issues/23563#issuecomment-2077777430