Feature idea: Add matplotlib backend

[joemarshall]

Right now, this is dependent on QT as a display backend.

QT is lovely, but very heavyweight. This limits where neoscores can be deployed, particularly they can't easily be used in notebook environments (e.g. jupyterlab, google colab), and also currently can't run on webassembly platforms (python in web-browser).

If it supported a more portable display backend then neoscores would be available in loads more places, you'd be able to have example notebooks etc.

The core scoring elements of this project use text rendering, drawing of lines, bezier curves and filled shapes.

One possibility of doing this would be to implement a matplotlib based backend. Matplotlib is supported pretty much everywhere python is available, is quite low technology, supports most features natively, and provides pretty good quality PDF, SVG etc. output, plotly.js output for interactive web rendering etc. It works in all python notebook environments that I'm aware of, including webassembly ones which run purely in browser without a server. QT doesn't yet work nicely in any of these environments, and PyQT is so heavyweight that unless both PyQT and QT maintainers decide that supporting notebooks is a high priority it is unlikely to happen.

As I see it, the minimum work to do this would look like:

1) Move the various bits in interface which depend directly on QT into interface.qt (e.g. various _create_qt_object methods, stuff in app_interface). i.e. so the interface.PositionedObjectInterface class and similar are abstract interfaces only, and filled in things in the QT backend. 2) Create matplotlib backend classes for shapes, lines. This is relatively straightforward, as matplotlib supports at least all the QT features neoscore uses here (lines, filled shapes, beziers). 3) Create backend classes for plain text. This would primarily be a case of fiddling with how font properties etc. are handled to ensure scores look the same, text is the same size etc., and is not such a big job. 4) Create backend class for rich text. This is non-trivial, because it would need to implement the QT subset of HTML, or at least as much of it as neoscore wants, and decent text layout isn't that easy. It would maybe be possible to hand this off in part to something else such as https://github.com/litehtml/litehtml although making sure the behaviour is analogous to the QT behaviour becomes harder then. One other possibility would be removing back-end use of rich text at all, and reimplementing a well defined subset of html in python, and only using non-rich text output in the backend. This would give consistent output between different backends.

[craigvear]

Great ideas Joe. Thank you. I’ll respond to your email next week, and grab a coffee sometime.

[ajyoon]

Thanks for sharing these ideas! As Craig will tell you, migrating away from Qt is already the next major roadmap item for Neoscore, for many of the reasons you articulate. I don't think we've considered matplotlib; it's an interesting idea for sure. My main point of concern would be interactive contexts, where we need consistent and high frame rates; matplotlib is generally not optimized for these use-cases, and in my experience has always been a bit choppy when used interactively. The free jupyter integration is a nice bonus, but I believe it's actually pretty easy to implement ourselves if we wanted to.

Another note - matplotlib itself supports several different graphics backends, and as such has an architecture similar to neoscore. One of those backends is actually a Qt backend!

[ajyoon]

I've noted homemade Jupyter integration appears to be low-hanging fruit in #108