Proposals

[jagenjo]

Here are some proposals with features I felt missing in the past:

• ctx.shapeSmoothEnabled = false; Allow to disable Antialiasing for primitives:
• ctx.shape( vertices_array ); Specify shape by passing a list of vertices instead of one by one.
• ctx.fillTriangle(v1,v2,v3, c1,c2,c3 ); Draw a triangle specifying 3 vertex and 3 colors, or even 3 uvs. Ideal for continuous surfaces.
• ctx.colorizeImages = true; Colorizing images by fillColor (currently I have to do hacks with globalOperation
• ctx.drawImage( webgl_texture_handler, x, y ); Being able to use a WebGLTexture Handler with drawImage (it already supports using a WebGL Canvas and most backends render canvas2D using OpenGL).

Nice to have

• ctx.transformCoordinate(x,y); a function that given a 2D coordinate it transforms it using current transform

[fserb]

Thank you so much for bringing those up. I'll bring those up next time we are looking for new features, but a lot of those make sense.

If you ever have other suggestions, please don't hesitate to bring them up. Also, if there are particular use cases that you think may not be covered properly, even if you don't have an actual feature proposal, those are also appreciated.

[jagenjo]

I think current proposals cover most use cases.

Some yeas ago I reimplementes the whole Canvas2D context in WebGL for a project where I wanted to mix Canvas2D and WebGL seamesly without having to upload the whole canvas to the GPU every frame. The project can be found here:

https://github.com/jagenjo/Canvas2DtoWebGL

And while doing it I started experimenting with extra methods that I found necessary as the mentioned before. Thats why I want to bring them to regular Canvas2D. Most of them are in your proposal (like matrix4x4 as transform, roundRect or fillColor).

Glad to help.

[Kaiido]

We'll probably come back to this later, but a few notes:

• ctx.shapeSmoothingEnabled = false; is definitely something that is wanted by web-devs. SVG already has shape-rendering, hopefully most engines would already have a path to it (I didn't check). Would be great to also include the ctx.shapeSmoothingQuality while we're there.
• ctx.shape() sounds more like a ctx.polygon() (each vertex is linked by a straight line right?). I feel this can already be tackled today quite simply with vertex[].forEach( ({x, y}) => ctx.lineTo(x, y) );, not sure a new method is really necessary here.
• ctx.fillTriangle(v1,v2,v3, c1,c2,c3) would be the only place where the context's fillStyle would not be used to fill a color, and it's not clear what c1``c2 and c3 would be (Only CSS colors? Any CanvasFillStyle including CanvasPatterns etc. ?). All in all, I'm not sure this is really compatible with the current API. What about a new CanvasGradient instead?
• ctx.colorizeImages = true sounds like it would do what either filters or gCO already can do, with less control to the web-devs and more "liberty" on the UAs (which could lead to more discrepancy across implementations).
• ctx.transformCoordinate(x,y) is ctx.getCurrentTransform().transformPoint({x, y}), I'm not sure we need anything shorter here.

[jagenjo]

My reply:

• about shapeSmoothingQuality, good idea.
• the reason to use ctx.shape (or ctx.polygon) is a performance reason, we complex shapes, calling thousands of times to lineTo has a performance penalty. I understand there is Path2D but sometimes the shape is dynamic and changes every frame and using Path2D has the same issue.
• fillTriangle is because some of us like to render 3D objects using regular Canvas2D where WebGL would be overkill, and fillTriangle is a function already supported in some engines like Pico8 or TIC80 that is very handy when rendering lowpoly 3D objects. And using a gradient is overcomplicated and way slower.
• colorizeImages was to make it more accessible for people who do not understand how filters work, but if it is easy to use filters to colorize an image, then I dont see a problem.
• about transformCoordinate I didnt know about transformPoint, if that is the case, then there is no need.

Thanks for taking the time to answer.

[Kaiido]

Regarding the calls to lineTo(), the performance impact of these calls is actually very minimal in comparison to their actual painting.

[my2iu]

fillTriangle would be great for supporting next-generation gradients. All these conical gradients and filters and whatnot are from 20/30 years ago, and better stuff is starting to become available now. Ideally, you should be able to specify different alpha values for each corner too. The performance would have to be good enough to support thousands or tens of thousands of triangles though.

If fillTriangle supported an image/texture like drawImage, it could be useful in 2d games for boned animation, particle effects, or simple lighting effects. I'm not sure if any 2d games still use canvas any more or whether they've all moved to webgl. In the past, people moved to webgl for supposedly better performance. I'm not sure what the latest opinions on that are.

I think it would be nice if drawImage supported pulling different frames of animation from a gif or animated webp or video. I find it annoying when making web games that we have to send animations as flattened sprite sheets instead of taking advantage of the better compression that's possible by sending them as animations or video clips.

Additional possible directions for canvas would be

1. make it more like a 2d webgl/webgpu with support for shaders and performance features so that it can be competitive for making 2d games
2. make it full-featured enough that it might even be possible to build a complete web page renderer with canvas itself. Notably, low-level text functionality has always been a little shaky in canvas. It's like a bootstrapping compiler. You know a compiler has a sufficient minimal feature-set if it can compile itself. Similarly, you know the canvas API is powerful enough for general use if you can use it to render the rest of the web page. That's not possible right now.
3. give up on canvas, but make a lot of canvas functionality available in webgl and webgpu. If developers need the extra power and performance of webgl/webgpu, then maybe canvas graphics model is simply out-of-date for modern needs. But some of the features of canvas should be made available in webgl/webgpu somehow, such as text rendering functionality and 2d paths. There should also be better support for bridging html/svg/videos into webgl/webgpu. For example, I should be able to take my entire web page, render it into a canvas, and then display that canvas in webgl so that I can interact with my webpage from within web VR.

[my2iu]

Just to give a little more context, the fillTriangle primitive is part of the PDF 1.3 specification from the year 2000 (look up the section on gouraud triangles), so it's a well-established 2d drawing primitive.

The idea of fillTriangle also accepting a texture/image as a parameter is more of a gaming thing, but it's a PlayStation 1 era graphics primitive, so it is also a well-known and proven drawing primitive. Even outside of gaming, it's useful for colorizing icons so that an icon can be white in dark mode, but black in light mode (though I suppose filter functionality could also be used to do that but in a more roundabout way).

[Kaiido]

the fillTriangle primitive is part of the PDF 1.3 specification

As far as I can tell by looking at this 21.4MB PDF specs, PDF path drawing commands are m, l, c, v, y, h, and re which respectively represent moveTo, lineTo, cubicBezierCurveTo, cubicBezierCurveTo, cubicBezierCurveTo, closePath, and rect (¶8.5.2)
The Gouraud-shaded Triangles meshes (Free-form type-4 and Lattice-form type-5) are Patterns (¶8.7), not primitives.

To make a parallel with the Canvas2D API, these would correspond to a CanvasGradient object. As I stated in https://github.com/fserb/canvas2D/issues/19#issuecomment-878008688 this indeed sounds like the best direction for this. I'm still not convinced that we need a new CanvasPath triangle primitive.

---

it would be nice if drawImage supported pulling different frames of animation from a gif or animated webp or video.

The WebCodecs APIs should handle this. For gif images there is the ImageDecoder, for videos the VideoDecoder one.

[my2iu]

I would argue that since gradient triangles and gradient textured triangles are the underlying primitive of a modern 2D graphics pipeline, it would make sense to expose them as fundamental operations in the API, and to make them performant too. If you make these operations only available through CanvasGradient, programmers will, essentially, draw a single big rectangle over the whole canvas and just draw everything using a big CanvasGradient that fills the rectangle. And it will be a mess because programmers will end up bypassing all the transforms and paths. It’s not clear if it will be performant either.

If you look at the way gradient meshes are exposed in vector drawing tools, they are exposed as their own primitive, even though they are exported as Patterns in PDF documents. They are like text and images—they are their own thing separate from paths.

[Kaiido]

I'm sorry I don't quite understand your arguments.
In PDF these Shading Patterns are fill styles, and they work almost in the same way as HTML's CanvasGradients do: They are clipped by the painted path fill-or-stroke area and are dependent on the current transformation matrix. Regarding the performance point, while I'm not an implementer myself, we can already observe that it's slower to fill numerous paths than it is to perform a single fill operation of a more complex path, (because the former implies more GPU draw operations), so I'd expect a big triangle-meshes-gradient to also outperform many triangles.

Also, while I guess there will be a performant path with HWA, I can also imagine that some will complain that in CPU rendering shaders aren't accessible and that it could be very slow (but once again I'm not an implementer and may be wrong).

[my2iu]

Yes, I'm assuming that a proper performant API would allow programmers to send batches of thousands of triangles in one go, either that or the rendering context would batch up triangle operations itself.

I'm arguing that in games and in modern gradient systems, all rendering is done with (textured) triangle primitives. In implementation, there probably isn't much difference between rendering triangles while clipping them to a path, or rendering triangles as a canvas gradient that fills a path. You could also argue that images and text should also be CanvasGradients because maybe a programmer wants to stroke a path using text and image patterns.

The main issue is that I feel like the API designers won't give enough attention to the design of the API if they hide it in a CanvasGradient. Graphics engines will want to render everything except for text using these (textured) gradient triangles. It's important that these triangle APIs are designed carefully to support the use-case of all rendering going through them. They have to be sufficiently flexible and performant to support 2d games with skeletal animation of its sprites (like Esoteric Software's Spine) or vector drawing packages that where entire images are made up of Coons patches and diffusion curves. You could argue that those graphics engines should simply adopt WebGL instead, but then I would suggest that Canvas can't meet the needs of modern 2d graphics engines and that the future of Canvas is to make its functionality more accessible from WebGL and not to add new features.