vilya / Shadertron

Run, record and edit ShaderToy shaders without a web browser or internet connection.
13 stars 1 forks source link

Cross-platform audio & video support #39

Open vilya opened 5 years ago

vilya commented 5 years ago

Qt's multimedia framework is a bit of a disaster. It's a wrapper around each OS's native multimedia APIs, so it gives different results on different platforms. For example on Windows you have to install a thirdparty codec pack to be able to play .ogv and .webm files; and on macOS I haven't yet figured out how to make it support those formats at all. The implementation is spotty too, with some things broken - a big one being that video and audio probes don't work on Windows (the bug report for that is marked as fixed in Qt 5.13, which isn't out yet).

Alternatives to consider:

vilya commented 5 years ago

An extra criticism of Qt's multimedia framework is that you have to do quite a lot of work to get video or audio data into a GL texture, both in terms of execution time and amount of code required.

In theory Qt can tell you when video data is already stored in a GL texture, which would enable us to use a high performance zero-copy path, but in practice it never seems to make this option available. This was the case with Qt 5.12; I haven't tried again since switching to the Qt 5.13 alpha.

It was pretty shocking to compare this with openFrameworks which makes the high performance path trivial to implement.