Evaluating fastuidraw for VCV Rack

Hello! I'm currently evaluating replacing NanoVG with fastuidraw in VCV Rack v3, scheduled for development in late 2020 and release early 2021. (Yes, we plan that far ahead.) I haven't seen a lot of talk about fastuidraw in the wild, but it looks beautifully designed, and at first glance delivers what it "advertises".

It would be greatly appreciated if anyone can answer any or all of these questions.

Does this project have any funding sources? (Intel funding?)
Does this project have a funding sink? (A place where I can throw money for maintainers to solve problems or push the project in some direction)
What is the future of this project? NanoVG's maintainer Mikko seems to have moved on, and while NanoVG can be adopted by someone (re-adopted by Mikko) with enough funding, my first impression of fastuidraw is that it may be years ahead of NanoVG's rendering performance and capability.
Who maintains this project? @krogueintel or a manager at Intel? I don't see a lot of discussions anywhere, so it seems like most planning is in @krogueintel's mind. (No problem with that at all. VCV operates smoothly this way.)
25-50 of the ~170 plugins for VCV Rack, each maintained by a different developer, call the NanoVG API directly (because I include nanovg.h in the Rack SDK), so functions as simple as nvgRect(vg, x, y, w, h) and as complicated as nvgRadialGradient(vg, cx, cy, innerradius, outerradius, innercolor, outercolor) would need to be wrapped to fastuidraw, to avoid breaking those plugins. Since both libraries attempt to follow HTML5's canvas API loosely, this would probably take at most a week. I guess this isn't a question but just a comment.
An open ended question: VCV Rack renders a huge number of paths per frame, and everything else is rendered to framebuffers to reduce this number (which are then rendered as images in the scene graph along with the other non-cached paths). VCV Rack runs on Mac/Windows/Linux and currently has graphics performance problems for users with weaker GPUs (e.g. pretty much any Macbook). Is fastuidraw an appropriate choice this project in your opinion?

Apologies if I come across as a bit pushy with all these questions.

Answers to some questions, first the technical parts:

25-50 of the ~170 plugins for VCV Rack, each maintained by a different developer, call the NanoVG API directly (because I include nanovg.h in the Rack SDK), so functions as simple as nvgRect(vg, x, y, w, h) and as complicated as nvgRadialGradient(vg, cx, cy, innerradius, outerradius, innercolor, outercolor) would need to be wrapped to fastuidraw, to avoid breaking those plugins. Since both libraries attempt to follow HTML5's canvas API loosely, this would probably take at most a week. I guess this isn't a question but just a comment.

This is going to be the biggest potential performance issue; FastUIDraw's path objects are quite heavy and should not be reconstructed every frame; in contrast NanoVG does not really have a path objects, instead all paths are created each rendering pass with nvgBeginPath(). In theory, one could wrap this up behind the scenes, but recreating the path every frame is a performance killer, especially for path filling. In FastUIDraw, a path's fill data is embodied by a FilledPath (see https://intel.github.io/fastuidraw/docs/html/d7/d9a/classfastuidraw_1_1_filled_path.html) or if the path is drawn smallish by a ShaderFilledPath (https://intel.github.io/fastuidraw/docs/html/d7/d86/classfastuidraw_1_1_shader_filled_path.html). Both of these objects are quite heavy to create, but once created fast to draw. In theory, one could use (or really abuse) Painter::queue_action() (see https://intel.github.io/fastuidraw/docs/html/da/de0/classfastuidraw_1_1_painter.html#a6f6b49e429d2d523a2be51f720e6ff37) to implement stencil-then-cover, but this prevents batching of drawing objects which in turn prevents a fair amount of how FastUIDraw gets good performance.

An open ended question: VCV Rack renders a huge number of paths per frame, and everything else is rendered to framebuffers to reduce this number (which are then rendered as images in the scene graph along with the other non-cached paths). VCV Rack runs on Mac/Windows/Linux and currently has graphics performance problems for users with weaker GPUs (e.g. pretty much any Macbook). Is fastuidraw an appropriate choice this project in your opinion?

FastUIDraw was developed on Intel first and should run fine with MacBook's Intel GPU's (even with Apple's terrible borking of OpenGL). However, it is critical that the paths are -static-, i.e. the same exact Path objects are stroked or filled, however the transformation (or even shader) used to draw the path can vary every frame without an issue.

Now for the non-technical questions

Does this project have any funding sources? (Intel funding?) Does this project have a funding sink? (A place where I can throw money for maintainers to solve problems or push the project in some direction) What is the future of this project?

I left Intel in July of this year as I got an amazing job from another firm; I work on FastUIDraw sporadically because, in all honesty, I had put too much of my heart into it to abandon it. While at Intel, with the exception of one manager, it was a constant fight to even get to work on on it (and even in that last case it was centered around making SKIA be able to use FastUIDraw).

Talking brass tacks, to get FastUIDraw to be a drop-in replacement for NanoVG, the following is needed to be done:

change the modified GLUtess inside of FastUIDraw (at https://github.com/intel/fastuidraw/tree/master/src/fastuidraw/internal/3rd_party/glu-tess) to use a memory pool allocator. This will make reconstructing filled paths many times faster than it is now
implement the needed code to allow for stencil-then-cover path rendering to be be batched up (so that the number of GPU state changes are amoritized)
change the image-atlasing in FastUIDraw to be faster in rendering; this either means killing GL_LINEAR_MIPMAP_LINEAR or changing it to a more traditional image atlas technique.

and to make FastUIDraw really great:

implement a Metal backend because Apple has deliberately borked their GL support
implement anti-aliased clipping
MAYBE implement caching of the coverages made from anti-aliased path stroking and filling.

Each of these I know how to do. If you are willing to fund me, I can try to work on them, but I need some permission from my current employer before I do this.

Thanks for addressing all this!

FastUIDraw's path objects are quite heavy and should not be reconstructed every frame

I'll look into this with my application. Most objects like UI buttons do have persistent state and exist in my custom Widget C++ class scene graph structure. They could theoretically store a persistent fastuidraw path object that would only be changed when needed. However, some objects would not able to do this as they are "re-pathed" each frame. We've always used HTML5 canvas-style path building, and everything operates under this assumption so it'll be a while before I can look into all of these differences.

implement a Metal backend because Apple has deliberately borked their GL support

That would be nice, because OpenGL performance keeps going down after each MacOS version, as they bork it more and more.

If you are willing to fund me, I can try to work on them

I've had funding set aside for improvements to NanoVG but haven't used it because I'm uncertain how much more can be done with its architecture. I'll have to evaluate my options which might take several months, but I'll keep you updated.

By the way, my Intel e-mail address is dead. Use my e-mail address that is in all the source file decorations of FastUIDraw

@AndrewBelt Any updates on your decision? I'm also currently looking into both FastUIDraw and NanoVG for a simple (mostly retained-mode) cross-plattform vector-based UI. How does the SVG rendering in VCV Rack actually work? Parsing the SVG during initialization with NanoSVG and keeping the information in main memory?

A big target for improving macos rendering would probably be supporting Metal, have you checked out https://github.com/ollix/MetalNanoVG? Not supporting Metal would be my main concern about FastUIDraw for cross-platform but I have very little graphics programming experience yet :/

Hi,

I was just browsing and saw your question today; I apologize for the massively delayed response. I had not heard anything. For users, here are my thoughts on NanoVG vs FastUIDraw. I want to emphasize that NanoVG is really nice before I go further. So here goes:

Paths. NanoVG uses stencil then cover to fill paths and then draws some anti-alias smudge along the path. In contrast, FastUIDraw triangulates the path. The good part for NanoVG is that it's technique uses less memory, there is very little caching (beyond tessellation) would help and thus dynamic paths are just as cheap (mostly) as static paths. The bad part is that if one is drawing LOTS of filled paths, the overhead of the pipeline changes and the additional fill rate used can become an issue. I also suspect that if a path has cancelling curves, it will render anti-alias smudge in places it should not. FastUIDraw avoids these issues (cost and potential inaccuracy on naughty paths) by paying the heavy cost of triangulating the paths, but if a path is reused it is a one time cost.
Clipping. NanoVG has very limited clipping support: a screen aligned rectangle. FastUIDraw supports clipping against filled paths (with any fill rule). However, clipping in FastUIDraw is not anti-aliased though.
Text Rendering. NanoVG has a simple classic way of rendering text: it generates a coverage texture holding a bunch of glyphs and just draws rects. This is fine as long as one does not zoom into the text (since the magnification will make it awful blurry). In contrast, FastUIDraw has some pretty nifty glyph rendering shaders to handle zoom and look good. However, glyph rendering in FastUIDraw is then heavier per pixel.
Library size. NanoVG wins hands down here. It is much smaller and simpler than FastUIDraw, Indeed, NanoVG essentially generates triangles to feed relatively simple shaders. In contrast, FastUIDraw heavily relies on shaders to do alot of the work to reuse attribute data. The upshot is that porting NanoVG to other 3D API's is a many, many, times much smaller task than FastUIDraw.
Customized drawing. NanoVG has a limited brush and that is it. FastUIDraw goes shader crazy allowing for those willing to write some GLSL code to do lots of effects.

On the subject of ports, FastUIDraw builds for emscripten targeting WebGL2. MacOS builds work fine too (read the README though). I have not tried iOS... and the WebGL2 won't work on iOS devices because Safari lacks working WebGL2 support.

@krogueintel thanks for the detailed overview and comparison! It sheds some light on very important aspects of the ideas behind FastUIDraw.

Btw. do you think the "compiler will offload it to GPU if guided by intrinsics" approach is/will_be more successful than "manually write clever mutually interoperable shaders with extreme data reuse and offer corresponding safe API" (which seemingly FastUIDraw relies upon)?

intel / fastuidraw

Evaluating fastuidraw for VCV Rack #61