Make Practical

[dcjones]

Ping @timholy. This may interest you.

This repo is a little experiment I just did to see if skia could get us better drawing performance than cairo. Skia was developed for chrome. Firefox was using cairo but switched to skia, I think for performance reasons.

I did a little benchmark (in src/bench.jl) drawing 1e6 randomly placed circles, and the times are promising:

Skia: elapsed time: 1.74048119 seconds (43 kB allocated)
Cairo: elapsed time: 6.268057011 seconds (2 kB allocated)

It wouldn't be hard to write a Compose backend for Skia and get significantly better performance on large plots. However major caveats:

• Skia does not have a package in most repositories (e.g. homebrew, debian, etc).
• It's intended to be statically linked to C++ programs. It took me a little tinkering to get a usable shared library.
• The ABI is not stable, and the API is only semi-stable.
• The C API is very incomplete and not intended as more than a toy. E.g. I had to add a draw circle function because they had only bothered with oval. Skia also has some fancy GPU backends but none of that is exposed in the C API.

So if we were to make this a practical library we would be forced to 1. maintain our own C bindings, 2. build and distribute our own binaries for the library. Maybe that's worth it, I'm not sure yet.

[timholy]

Thank you! I'm now watching this repository. I'm really hoping to be done with Array-related stuff by summer, and it would be fun to get back to graphics.

[timholy]

BTW, there are two other possibilities:

• see if we can upstream our C binding, so it's no longer our maintenance worry
• use Cxx.jl, once that can be used on off-the-shelf master

[dcjones]

It did occur to me that this might be a job for Cxx.jl. I'll give it a shot and see how far I can get.

[timholy]

Last time I tried Cxx, it required a version of LLVM that nixed your backtraces. To me that definitely wasn't a worthwhile tradeoff. But perhaps the situation has improved, I seem to remember seeing some progress there.

[timholy]

Also, I'm not exactly sure how you're planning to incorporate this, but if you're thinking about wrapping a C++ library, do keep QPainter in mind. When I tried Qwt (which is surely based on QPainter), it too was considerably faster than Cairo. Moreover, Qt obviously offers a lot of functionality.

[dcjones]

I don't have definite plans, I'm just feeling out possible paths to higher performance raster graphics. Qt may be a better bet than Skia if it's competitive in performance, since at the very least it's an easier dependency to satisfy. Though it shares with Skia the dilemma of either using Cxx.jl or writing our on C bindings.

[timholy]

OK. I also wasn't sure if Skia lets you render directly to a browser window or something, which would obviously be a killer feature. To the best of my knowledge that's not possible in Qt.

To my mind, another potential killer feature of Skia would be if you can render to a GtkCanvas, since that would let us leverage @vtjnash's awesome work on wrapping Gtk. (Despite having once lobbied for Qt, I'd strongly prefer to stick with Gtk given that it's here and a Qt wrapper would be a major undertaking.) We already have that with our current Gtk/Cairo combination, but as you note Cairo's performance is very worrisome. Using Skia with GTK looks promising.

Finally, there's always the render-straight-OpenGL approach that I suspect @simondanisch would endorse. My concerns there are (1) if possible, it sure would be nice to have a GL-free fallback, and (2) I worry that incorporating plots into GUIs (like ImageView) may be painful if that's always limited to a GLFW window. I'd much rather have a plotting canvas inside a Gtk (or Qt, if necessary) window I can render to, so I get to use the maturity of those widget toolkits. For a long time, it seemed that it was a lot of effort to build a GL extension with Gtk, but I just googled and came across this happy (and relatively recent) announcement: http://anzwix.com/a/GTK/GdkAddSupportForOpenGL If that works well, it's also something to consider.

[SimonDanisch]

Hi there, I got around to make a little prototype:

It's also 1e6 randomly placed circles, which draw at 123.72704ms per frame, which means you can interactively zoom and pan around. The camera can use a 3D projection, but can also be orthographic ;) (This is on a crappy Intel i5 + HD 4400 platform) It's a pretty limited approach which only works for simple shapes, but I'm pretty sure that it can be extended in some way (It's a hard coded distance function, which can be replaced by e.g. some stored distance field or a texture). We're not set on GLFW, all that is needed to port this to other platforms is the context creation and pumping the events into Reactive signals. When you can do this with QT or Gtk, you could easily draw on their context. Web seems a little more complicated, as you have to go through javascript in some way, from what I understand. Never researched this, as I'm not too fond of going down this path... I made a big update to GLVisualize recently (which will soon be the core of GLPlot), but sadly, I didn't ported my widgets to the new architecture in time (i've had sliders, textfields and color choosers implemented) , and I really need to go back to my thesis now... So no native GUI's yet ;)

I did some more research, and what is really difficult is drawing splines (duh!). As long as we can make compromises around this, it seems there are tons of options to get simple yet effective 2d drawing on the GPU.

For the OpenGL free fallback, I hope that it will be possible to have some intermediate representation which can be easily sent to different back-ends... But doing this gracefully seems a little bit hard, especially with my demands for interactivity and time changing signals. Well I'm not going to think much further about this, until I have something stable going with GLVisualize... Before GLVisualize doesn't reach some maturity and proofs, that my concepts are somewhat sound, it doesn't make much sense to freeze anything.

[jcgregorio]

Hey, glad to see you're looking at Skia for the rendering!

"I had to add a draw circle function because they had only bothered with oval. Skia also has some fancy GPU backends but none of that is exposed in the C API."

Those would be fairly straight-forward additions we could do.

Just as background, we are moving pretty slowly on the C API because we want to build something that is stable and maintainable over the long haul, thus all the warnings right now about the code being experimental. If you do find other missing functionality in the C API please file feature requests (https://bugs.skia.org) and feel free to also shoot me an email (jcgregorio@google.com) so I can properly flag those requests as being for the C API.

I'm also curious how you built Skia and if you tried our also very experimental CMake build:

https://github.com/google/skia/tree/master/cmake