Fit lyon into a 2d game engine

[Noxivs]

Hello gfx rusters :)

I am currently trying to find a solution to convert and render flash animations within Webrender :

• The conversion step is to export every shape in a SVG file and to list every transformations for each frame in an other file.

• The rendering step is to rasterize the shapes and to apply the transformations.

Lyon and Pathfinder/Partitioner (from @pcwalton) seem to fit the job of the rendering step.

I don't have any experience in GFX development and if you had a moment I would have some questions.

1/ Could lyon be a good alternative to render flash animations (2d game) ?

2/ Would Pathfinder/Partitioner be more appropriate ?

(Pathfinder/Partitioner is still in development to support full scene rasterization and the main goal is to render font.)

3/ Pathfinder/Partitioner provides high quality antialiasing. The AA of Pathfinder/Partitioner could be used for lyon ?

4/ Fit lyon into servo/webrender is an official purpose ?

(My final goal is to use servo or webrender for the user interface over my own "game map render" which uses currently webrender.)

I apologize in advance if my questions are not insightful, but It would be very helpful :)

[nical]

Hello gfx rusters :)

Hi!

I am currently trying to find a solution to convert and render flash animations within Webrender : The conversion step is to export every shape in a SVG file and to list every transformations for each frame in an other file.

The rendering step is to rasterize the shapes and to apply the transformations.

You mean rasterize each shape in a separate texture and then composite them together? If so, this approach will probably be tricky to get to perform well with complex drawings (and use lots of memory). But it is also a simple approach to implement so it has its merits. If this is what you are going for, ignore what I am going to say about pathfinder it will kinda work if the rendering is happening this way.

I don't have the time to describe more optimal approaches to rendering with lyon/partitionfinder right now, but I'll come back with more info if you are interested.

Lyon and Pathfinder/Partitioner (from @pcwalton) seem to fit the job of the rendering step.

Lyon and partitionfinder are trying to solve (the hardest part of) the problem of rendering complex paths on the GPU (think "paths" as in svg/flash drawings). Pathfinder is more geared towards font rendering (it is particularly well suited for small shapes with not a lot of overdraw.

Partitionfinder is a what lyon version 2 will look like some day (in fact the two projects could merge at some point), that is a resolution-independent tessellator with a slightly more involved rendering system that does not need to re-generate geometry if you zoom in and out. Lyon, however, does the more naive thing (that almost all other path tessellators do) to generate geometry for a certain zoom level by approximating curves with a fixed number of line segments.

1/ Could lyon be a good alternative to render flash animations (2d game) ?

Yes, kind of. Lyon implements a piece of the 2d rendering pipeline that you will need to implement. Lyon only generates the tessellation (the set of triangles that exactly covers a shape). Then it is up to the engine using lyon to implement anything related to actually rendering it (how to do gradients, colors, textures, etc. A very popular flash gpu rendering system called scaleform (which was shut down very recently) was using a similar approach (using a tessellator) to render vector graphics. So the idea of using lyon for this type of thing is reasonable. What I want to make clear, though is that lyon is just piece of the puzzle, and while I am not familiar with the featureset of flash, there might be a lot more to implement.

2/ Would Pathfinder/Partitioner be more appropriate ?

Partitionfinder sits at the same layer of abstraction as lyon (it's not a rendering engine, it's mostly a tessellator that gives you geometry for a path and you are left with the task of rendering that geometry). It is more ambitious and in a more experimental state than lyon. So if you use it, you get resolution-independent tessellation, but by the time you have integrated it in your project, Patrick might have rewritten it several times from scratch.

Pathfinder is, in my opinion, not as well suited for general purpose drawing (but better suited for font rendering). I wouldn't use it for a general purpose flash drawing primitives.

(Pathfinder/Partitioner is still in development to support full scene rasterization and the main goal is to render font.)

Not exactly: Partitionfinder's main goal is general purpose drawing primitives (SVG, canvas, etc) just like lyon. Pathfinder, however, was designed for font rendering.

3/ Pathfinder/Partitioner provides high quality antialiasing. The AA of Pathfinder/Partitioner could be used for lyon ?

Patrick is still figuring out antialiasing for Partitionfinder (but in a good place to get there). Lyon has no anti-aliasing at all right now (ouch), it would be easy to add aa for strokes, and it would be great to also do it for fills but it'll take a bit more time for me to get there due to the amount of time I can dedicate to the project. Another way is to not do the aa in lyon, and approach anti-aliasing the way game some engines do (temporal reprojection, screens-space aa, etc.). Or use MSAA but it's not very good quality.

Pathfinder has very nice aa, but again, I don't think it would be the best for what you are trying to do. It isn't possible to take pathfinder's aa and put it in lyon because these two projects are very different in their rendering approach.

4/ Fit lyon into servo/webrender is an official purpose ?

Let's put it that way: I work on webrender in my day job, and lyon is a side-project. I would love for it to end up in webrender, but it's not quite there yet in terms of featureset (it's good enough as a prototype for use in servo, but not for firefox).

Patrick has the advantage of working full-time on partitionfinder so he'll probably get to something shippable sooner than me. when that happens I hope that the two projects can merge.

[nical]

Pathfinder/Partitioner is still in development to support full scene rasterization

Just to clarify. Flash has a concept of full-scene rasterization (and scaleform had something sorta similar), but the term is usually used to refer to their sweep line algorithm that renders all of the shapes at once instead of rendering them one after the other (scaleform's equivalent is a "full-scene rasterizer" in the sense that it generates a tessellation of all of the shapes at ones with no overlap between opaque triangles).

Lyon, pathfinder and partitionfinder don't attempt to do that.

But I am not sure if by full-scene rasterization you were referring to flash's algorithm.

[Noxivs]

Thank you a lot for these clarifications ! I didn't expect that.

You mean rasterize each shape in a separate texture and then composite them together?

Exactly.

I don't have the time to describe more optimal approaches to rendering with lyon/partitionfinder right now, but I'll come back with more info if you are interested.

It would be very nice for me to have your opinion on the subject one of these days !

Then it is up to the engine using lyon to implement anything related to actually rendering it (how to do gradients, colors, textures, etc.)

Lyon will not implement these kinds of features to fully render a svg file (as an extra library) ?

So basically, in order to get a first runnable demo which renders converted flash contents, I will have to contribute actively to lyon. I guess I will probably start with the easy labels :smile:.

[nical]

It would be very nice for me to have your opinion on the subject one of these days !

When I have some time I'll try to do a writeup on how to get the most of lyon (and other tessellation-based rendering tools). In a nutshell, lyon tries to make 2d rendering similar to 3d. That means taking vector paths and turning them into vertices and indices that describe geometry the same way we are used to representing 3d models. The rest of the rendering pipeline becomes pretty much like a 3d engine, with the same type of optimizations: batch as many paths together into the same vertex and index buffers to submit them in as few draw calls as possible, render opaque geometry front to back using the depth buffer to avoid overdraw, then transparent geometry back-to-front, etc. The idea is that GPUs are optimized for rendering 3d scenes. So treating 2d rendering the same way we do 3d lets us get the most of the silicon that was designed specifically for that purpose, and lets us benefit from years of research in optimizing 3d rendering engines.

Lyon will not implement these kinds of features to fully render a svg file (as an extra library) ?

Full SVG support is a lot of work. If I was working full-time on this it would be on the table, but with the time I can dedicate to the project, it would just take too long. So I'd rather not advertise for something that ambitious since the time it would take to deliver is beyond what I am comfortable planning for in the context of this project.

For now, the plan is to focus on stroking and filling (even-odd fill rule) paths. I am experimenting with a slightly higher level library on top of lyon that will manage basic patterns (solid color, image, linear gradients), and some rudimentary batching, and maybe some limited form of clipping. That means no text rendering, no filters, no winding fill rule, no dashing, etc. in the short/medium term (unless someone offers to implement them). Hopefully, flash has a more limited set of drawing primitives than SVG.

WebRender will eventually implement all or most of SVG, using parts of partitionfinder/lyon or whatever else. But that also will take time.

So basically, in order to get a first runnable demo which renders converted flash contents, I will have to contribute actively to lyon. I guess I will probably start with the easy labels 😄.

Great! <3 Let me know if I can do anything to help you get started.