Investigate switching to Gstreamer

[RomanHargrave]

For a few number of reasons, I believe that it would be beneficial for tomahawk to adopt Gstreamer as its audio engine, replacing VLC. Such reasons are as follows.

1. This would make it very simple to support replaygain, as Gstreamer ships with a replaygain plugin that is easily added to the output pipeline

2. This may potentially reduce the amount of work required to support gapless playback

Looking at AudioOutput and AudioEngine it appears that replacing VLC would be a relatively straightforward operation, as LibVLC usage is effectively confined to AudioOutput.cpp. Packaging for Windows and macOS should continue to work as Gstreamer is available for both platforms.

This is something I am willing to work on, and may eventually open a PR for should I decide to do so.

[hugolm84]

Hi @RomanHargrave, thanks for your interest in contributing to Tomahawk!

libVLC used today is an pretty old version (v2) and much have happened since its release, both for GStreamer and VLC. That said, I'm not particularly updated with the current state of any of them.

Have you investigated if libVLC v3 or v4 provides the features that you are interested in improving? I think it would be a good idea to replace libVLC if the replacement offers something that libVLC does not.

That said, yes, it should be relatively straightforward to replace the underlying audio backend. Just make sure that streaming, rtmp, mp3, flac, seeking, duration, http works. Ideally, https streaming should work too (certificate validation/non-strict option).

It is not sufficient that GStreamer is available on all platforms, Tomahawk installers need to bundle the correct binaries. A potential approved PR would need to include this before merge.

Developers of Tomahawk are still active reviewing PR's and we would happily comment and give direction.

Best regards

[RomanHargrave]

@hugolm84 that's a good point with respect to libVLC - I'll have to look in to what's changed since v2. Your point about bundling binaries is a good one, and something I neglected to mention in the initial issue. It looks like it's pretty easy to bundle VLC for Windows, so I can see the value in sticking with it.

I'll take a look and see what I think.

[RomanHargrave]

Skimming through VLC issues and the libVLC documentation, it is somewhat apparent that VLC is not suited for gapless playback. In order for the underlying media player to be able to support gapless playback, there must be some way to prebuffer the next track. libvlc_media_player_t operates on a single track at a time and does not appear to support any sort of preloading functionality. I am going to test libvlc_media_list_player_t to see if it will preload media, but I don't have high hopes as the larger VLC project has had an open issue on the matter of gapless playback for about 14 years.

[hugolm84]

Skimming through VLC issues and the libVLC documentation, it is somewhat apparent that VLC is not suited for gapless playback. In order for the underlying media player to be able to support gapless playback, there must be some way to prebuffer the next track. libvlc_media_player_t operates on a single track at a time and does not appear to support any sort of preloading functionality. I am going to test libvlc_media_list_player_t to see if it will preload media, but I don't have high hopes as the larger VLC project has had an open issue on the matter of gapless playback for about 14 years.

Hi, I just made a quick search and it seems like vlc 4.0.0 supports gapless. See http://ftp.belnet.be/mirror/FOSDEM/2019/H.1309/media_vlc.webm @5:50 See the last comment in the issue you posted. Perhaps this merits some more investigation?

[RomanHargrave]

The problem, currently, with VLC 4 is that it's not even in Debian Unstable nor Arch at the moment (both have 3.0.X) - meaning that, while packaging for Windows would not be an issue (because the needed VLC binaries are included), distribution for Linux would be nearly impossible as VLC 4 is likely only to be present on those systems where the operator has compiled and installed VLC 4 themselves.

Looking at the libVLC documentation, again, it does not appear to have been updated to reflect the upcoming simplification of the media list API discussed in the FOSDEM talk link. Based on this alone, I'm unsure when to expect VLC 4 to be generally available.

[hugolm84]

If we assume that vlc-4.0.0 will be released in near feature and that it will be available on most distributions, you could start development on a branch that uses vlc nightly builds (https://nightlies.videolan.org/) locally. They seem to be available for manual install on most platforms or via apt for ubuntu (https://launchpad.net/~videolan/+archive/ubuntu/master-daily).

This would at least give you the option of evaluating vlc-4.0.0 and be ready for when it becomes available via package managers.

If you are still interested and think this is a good idea, I would be happy to assist should there be any need. I would suggest to use the tomahawk Docker image to "snapshot" the nightly build when building for linux or cross compiling for windows.

[RomanHargrave]

I'll investigate this.

[RomanHargrave]

After playing with some hacks involving the media list player and wasting time debugging inexplicable threading behavior in VLC's decoder loop, I've gone through and investigated the VLC 4 API and have the following to note

• Next-track discovery in VLC 4 is callback based and would effectively require some structural changes to AudioOutput and AudioEngine. For starters, the callback implementation would need to know - at the time of calling - what the next track in line is and be able to return it then and there. This is somewhat opposite to Tomahawk's current design where AudioOutput emits a signal that is received by AudioEngine, which then proceeds to load the next track into AudioOutput. This works with VLC 3's API design, and could work with VLC 4's design if there was some function to call in order to set the next track; however there is not. Instead, the callback approach requires a synchronous/transactional design where AudioOutput must either be aware of the upcoming file at all times or implement a system where the VLC callback waits until AudiOutput has resolved the next track. Because the threading semantics are not very well described, it is impossible to know how the latter approach would affect playback.

• Browsing the commit history, no mention of gapless playback being effectively implemented has been made, nor is it discussed in the NEWS file. Mention of the functionality is made once in git history (commit a80c78fc2a0431a8822e581126e108266ce07d9a in 2018), where it is stated that the framework is now in place to implement the functionality but that gapless playback itself is not yet implemented. While it is certainly possible in the hundreds (if not thousands) of commits since then that it has been implemented without any mention, I am not presently interested in digging in to the playback/decoder implementation to make this determination for myself. As it stands, the issue requesting this feature has not yet been closed in trac, and is part of the 4.0.0 milestone so I can only assume that it is not yet present (regardless of what may have been said at fosscon).

• The VLC 4.0.0 milestone is currently eight months overdue and is only 58% complete as of writing with a backlog of unresolved issues such as the gapless playback issue. I have been unable to find discussion of VLC 4 packaging on the Debian MultiMedia team mailing list, and there has been no (easily discoverable) discussion of a new release target date. While testing and development with docker is practical, day-to-day use is not. Without a release date or explicit confirmation of the desired feature's existence, beginning work on an exploratory VLC 4 playback backend seems almost entirely unwarranted even in the light of straightforward Windows support.

• If you look at the gstreamer engine in Strawberry/Clementine, you'll find that gapless playback is pretty straightforward and is implemented in a manner that's far more compatible with the signal-based design present in Tomahawk. The backend emits a signal when it nears the end of a file, which results in a call back to it's preload method with the track URI. The track URI is then passed in to the Gstreamer discoverer. That's it - Gstreamer handles the rest. There's no excessive transactionality, no callbacks, and no IPC/locking needed. This is certainly the direction that VLC is headed with VLC 4, but it's not absolutely there yet and I'm quite frankly not sure when it will be considering that VLC 4 is currently stuck in what appears to be development hell.

Gstreamer includes some documentation on windows deployment; however, Windows is not my area of expertise. In order to retain my sanity, I'll probably start looking at replacing VLC with Gstreamer, and if it's simple enough for me to package it for Windows I will. Gstreamer has a couple other hat tricks, such as simple cross-fading between tracks if you so desire.

VLC is not a bad project, but in my experience it is primarily focused towards codec support and standalone video-related features. Looking at libVLC 3, it seems to me like the embedding functionality was only ever designed with a handful of (very simple) use cases in mind. VLC 4, whenever it's finished, seems like an effort to correct the extremely anemic libVLC API and includes better documentation as well; however, owing to the aforementioned focus on the standalone aspects as well as codec support I don't foresee those features which are better suited to embedding in jukeboxes getting enough attention. As is the nature with open-source projects, features and improvements typically reflect the wants and needs of people willing to contribute (and this is not a bad thing). For VLC, it seems that contributors rally around decoding and user interface concerns because that's what they use VLC for - not complex audio playback. Gstreamer, on the other hand, has a lot of features and design decisions that make it an excellent choice for building audio playback (and recording) software.

[hugolm84]

Hi @RomanHargrave

Thanks for your efforts investigating this. I have also done some investigation and implemented a working audio backend using LibVLC 4.0.0.

Vlc is very competent as this works seemingly without issues, requires minimal effort to implement and libvlcpp framework simplifies a lot of the code. My takeaway is however that vlc 4.0 is not at the point where it can provide gapless playback nor preloading without significant work. See PR #583 for possible direction on this.

My view is that we should use the best tools available to provide the audio experience we are looking for. This being said, I think it would be warrented to evaluate if gstreamer can provide this instead.

Packaging should not be a big issue...

I suggest you make a PR early on if you like to be reviewed along the way. Please join us in #tomahawk @freenode if you like.

Best

[RomanHargrave]

Oh, I have no doubt about VLC's competence - and the 4.0 API is very nice. I just get the impression that they're a little starved of resources and manpower right now, given that they've got an apparently large backlog and are quite overdue on their roadmap.

I actually have looked at #583, and at risk of sounding a tad rude it doesn't seem like a very robust approach to the problem. Though it is a novel and effective solution.

[hugolm84]

I welcome your PR.

[dschmidt]

Nice, amazing research you've done there. Thanks a lot!

[RomanHargrave]

FYI - This is something I intend to work on but some stuff has come up at work/personal that may slow that down, just know I'm not abandoning this.

[hugolm84]

No worries, it takes the time it takes :)