Support Linux?

[YeungKC]

Is your feature request related to a problem? Please describe. At the moment it looks like support for window is in progress, if Linux can also be supported then great~~

Describe the solution you'd like n.a

Describe alternatives you've considered n.a

Additional context n.a

[maks]

Tying this to the windows support issue is where the dart_vlc plugin from @alexmercerind has been discussed to support windows and it now has suport for linux too.

[ryanheise]

Since I personally use Linux, I may personally give this a try (no ETA, though, since I'm also working on several other issues - if someone wants to jump in before me, advertise your intent both here and on the windows support issue so that we don't duplicate effort.)

[GZGavinZhao]

I'm thinking about using ALSA library with FFI to support audio on Linux.

[maks]

@GZGavinZhao no need to think about it, I've got a package already with simple playback working, PR contributions most welcome. Sorry @ryanheise I forgot to report back here when I started working on it.

If you'd like something more cross-platform, I can recommend the libao package which I've been happily using recently.

[GZGavinZhao]

@maks That's awesome! I have thought about libao before, but I think it would be better if we either just stick to separate frameworks for each platform or use a unified cross-platform framework, otherwise things might get messy. Since those who have been working on the Windows and Mac OS compatibility seem to already found their choice (correct me if I'm wrong), alsa might be the most suitable one for now.

Also, I'm a little bit worried about libao's support because it has been pretty long since they've had a new release.

[ryanheise]

Options options options!

There is no problem having multiple implementations in a federated plugin framework.

In #103 we had come up with a naming convention like this:

just_audio_library_platform

For example:

just_audio_libao_macos
just_audio_libao_windows
just_audio_libao_common

From the libao page it looks like there are multiple targets for linux, but we could still fit them into the above naming framework by treating oss, pulse and alsa as different platform targets (unless anyone has other suggestions?)

[maks]

My requirements for are for realtime audio synthesis, sample-playback and filter application (delay, reveb, etc) which I think are very different from just_audio's focus on media file/stream playback, compressed format support, metadata management etc, so I'm using Alsa and Libao directly via my packages, but like I said I'm happy to accept PRs that add whatever functionality would be needed to for it to be used by this package for linux support. But again given the rich feature set that just_audio already supports, I think you may find that using something like MiniAudio would work better, given its support of many of those features, even if you don't need it's multiplatform support, even though on Linux Pulseaudio is now the most widely used audio server, direct Alsa use is much less common and Jack is widely used for music-production low-latency applications.

[maks]

@ryanheise yes that naming sounds good, let a thousand flowers (platform implementations) bloom! 🙂 👍🏻

[GZGavinZhao]

Was able to get miniaudio to work under linux with dart:ffi, though I still need to try out how to asynchronously call the methods. miniaudio itself claims that it's asynchronous, so I guess there are some problems I need to figure out on the Dart side.

[alexmercerind]

@GZGavinZhao that's cool, but what's easiest is to map dart_vlc to just_audio's platform channel interface.

You guys don't need to worry about playback backends (or anything native), dart_vlc is being maintained by me & a lot of time has been already spent making it stable & fixing issues reported by the users. dart_vlc is also part of awesome-vlc page from one of libVLC community members here.

You can just add dart_vlc as a dependency to just_audio & use it (submit a PR to this repo). I'm also using FFI in this project now after migration from platform channels (until that time I couldn't figure out async callbacks). Just using FFI wasn't enough, I have also added event callbacks for position, video frames etc. through NativePorts of Dart VM. You just need to add Dart mapping to dart_vlc for just_audio.

If you are not statisfied with dart_vlc's interface, you can use baremetal C-like dart_vlc_ffi package that dart_vlc internally uses (however then you have to setup your own cmake script in just_audio to build dart_vlc.dll). It also has callbacks support.

Speaking of miniaudio, alsa or libao... Those libraries are really really low level, even writing network streaming from scratch will give us a lot of pain. libVLC already offers a nice interface & dart_vlc brings its most features to Dart. Audio playback is one of the features that we're offering & you can use it in just_audio.

@megamegax spent bit of time earlier trying this. But hasn't done much after that. https://github.com/megamegax/just_audio

And, let me know in the issues of dart_vlc if you guys face any kind of problem or issue. I'll be fixing them as best as I can.

Windows support for just_audio: https://github.com/ryanheise/just_audio/issues/103 (also can be targetted by same work)

Thanks.

[alexmercerind]

For Windows, I have even better idea to use C++/winRT APIs themselves (Windows 10+ specific however, I don't know what to say to people who use < 10) in future. Why deal with libao, libao buffers, decoding & all... (So, I'm working on an another separate side side side project to expose those APIs in a DLL, so that Dart VM can use it). It is going to be very light (while still having a lot lot better media codec support & features compared to miniaudio) since there will be no third party library used for playback but core Windows APIs. But, for now dart_vlc will be cool.

miniaudio is like a dead-end at this point unless one is really really geek to build things like network streaming, better format support on top of it. Speaking of miniaudio, I already worked on it (possibly world's first) to play audio on Flutter desktop, flutter_audio_desktop, but dart_vlc is just better.

[GZGavinZhao]

@alexmercerind Wow thanks for the suggestions and resources! I'm checking it out now. One concern I do have is about external dependencies (which was why I first tried miniaudio). When using dart_vlc, do we need any dependencies on the client side? I know that for developers we need the vlc development libraries, but when we build a Flutter app and distribute it to the users, do the users need to install anything on their side (perhaps something like libvlc)?

[alexmercerind]

@GZGavinZhao on Windows, I provide the libvlc source directly within the plugin & all the shared libraries are automatically copied with CMake (I have everything ready already, don't worry about that).

On debian, one must have vlc & libvlc-dev installed with apt (I believe depending on distro's repos is best bet, building libvlc can take ages). Someone has also added info to build it on redhat based distros. Not sure about arch however (although people have built it successfully on manjaro). Many distros like KDE neon (Plasma is on the rise) come with preinstalled VLC infact. I have written everything in the README that you might need to know about it.

[GZGavinZhao]

@alexmercerind Understood. Thanks for the detailed explanation!

[maks]

@GZGavinZhao great work getting miniaudio working, do you have a repo with that FFI binding? I started doing a package for FFI binding but didn't get very far before switching to Alsa and then libao.

I'm not sure why @alexmercerind you said "FFI isn't enough"? You just need to call sync C APIs like libao in a isolate to ensure you don't block the main isolate. I've done a simple implementation of that recently for my project and I'll write up an article on how to do that soon.

And yes with respect to libvlc you will need to ship it as a shared lib in your Flutter app to your users and be aware that while libvlc is LGPL licenced some modules maybe be GPL so that would mean your app then needs to be GPL too.

It's all really dependent on what you need. I chose to use Alsa and libao (miniaudio on my to-do list too) as I do need low level, sample level API access. If you instead only want a higher level API with features like streaming or compressed formats then something like libvlc is probably better choice which it might be given that just _audio seems to be more focused on those use cases.

[alexmercerind]

@maks

I'm not sure why @alexmercerind you said "FFI isn't enough"? You just need to call sync C APIs like libao in a isolate to ensure you don't block the main isolate. I've done a simple implementation of that recently for my project and I'll write up an article on how to do that soon.

Yes, FFI is not enough unless they fix this: https://dartbug.com/37022.

Its not about sending messages to C/C++ from Dart, but its also about being able to receive messages from C/C++ to Dart. miniaudio in comparison to libvlc or winRT APIs is very basic, there isn't much like event callbacks.

In just_audio, there are many event streams (as Dart has those great things) e.g. for position, playlist & other events etc. etc. (github.com/ryanheise/just_audio/blob/e312da2a210ac7b5c0c4ba85cc59c2c832cff52b/just_audio/lib/just_audio.dart#L301-L404) FFI is enough, if you just need to talk from Dart to C/C++, but not the other way around. And I'm not sure how will you be able to feed any Stream with values being returned from C/C++ asynchronously. Its a bug in FFI itself.

See how I had to use NativePorts to send back position, video frame buffers, playlist events etc. etc. asynchronously. https://github.com/alexmercerind/dart_vlc/blob/master/ffi/native/eventmanager.hpp because FFI wasn't enough.

Now one might say why use Stream or async callbacks, just leave it... We can't because that's how a Dart library is made, all developers are familiar with Stream subscriptions & its the easiest way to update the UI. Thus, for notifying about events being happened asynchronously, its necessary to send updates back to Dart. More on what problem is here in this comment: https://github.com/ryanheise/just_audio/issues/103#issuecomment-798848254. In C++/winRT aswell, one can register for event callbacks by passing a lambda (and that will get fired when certain thing happens), but using that in FFI will certainly result in a crash or use NativePorts.

And yes with respect to libvlc you will need to ship it as a shared lib in your Flutter app to your users and be aware that while libVLC is LGPL licenced some modules maybe be GPL so that would mean your app then needs to be GPL too.

libVLC can be compiled with only LGPL modules enabled. And difference will be unnoticeable. If one doesn't even wanna compile, they can just delete the shared libraries.

It's all really dependent on what you need. I chose to use Alsa and libao (miniaudio on my to-do list too) as I do need low level, sample level API access. If you instead only want a higher level API with features like streaming or compressed formats then something like libvlc is probably better choice which it might be given that just _audio seems to be more focused on those use cases.

That's what I also believe. For just_audio, general high-level interface to audio playback is required with features like network streaming, good format support, event callbacks etc. and its just hard to do in miniaudio or from bare alsa/libao & if one decides to do it themselves (it might be another media player from scratch...), it will be far from just a "binding" or a "plugin".

[maks]

@alexmercerind

Its not about sending messages to C/C++ from Dart,

Thats correct, because we are talking about FFI not platform channels, so we are not sending any messages, instead with FFI we are making function calls.

I don't know what API libvlc exposes, but Alsa and libao have synchronous APIs so theres no need for async callbacks from native code into Dart.

As I said you simply need to use a seperate Isolate that then exposes an event stream to ihe main Isolate. You don't give up on using Streams, you simply use standard Send/Recieve ports that Dart provides for all interisolate comms. The main isolate can then expose whatever streams are required using messages coming in from second isolates receive port. Here is a gist with code extracted from one of my recent projects that demonstrates the basic of this approach. My code is using sample based playback with libao, so its trivial to have this code report a playback stream in the main Isolate.

[GZGavinZhao]

@maks Thanks! I wouldn't call it a proper binding though, because technically I didn't wrap the miniaudio.h; I only bound the example here to Dart.

@ryanheise Just found a problem: because just_audio depends on audio_session, which doesn't have a Linux implementation, implementing just_audio for Linux would also mean implementing audio_session for Linux. If I don't do that, just_audio won't run and will throw something like:

No implementation found for method getConfiguration on channel com.ryanheise.audio_session

Any work-around for that? Like creating a temporary empty implementation?

[maks]

@GZGavinZhao a vaguely remember that audio session type functionality is available via DBus on Ubuntu and many other distros and there is already a DBus package available written by one of the Ubuntu staff developers.

[ryanheise]

Hmm, I thought I had already implemented audio_session as a noop on other platforms. That is something I should fix:

https://github.com/ryanheise/audio_session/issues/34

[ryanheise]

OK, this is now implemented in the git version of audio_session but I'll hold off doing a release until we can test whether this solves the issue.

[ryanheise]

Hi @megamegax I was just taking a look at your code and it looks like quite a clever approach! Basically, to start with the existing web implementation as a base, since it already implements all of the backend logic, and then just substitute in the vlc player instead of the HTML5 audio element.

Of course eventually we could reimplement the backend logic using vlc features like playlist, although that could be a substantial amount of work.

Are you interested in continuing on your project and/or collaborating? Or would it be best to work on it as a sub-project of the main just_audio repo? The idea of the federated plugin model is that in theory anyone can create an independent platform implementation and publish it themselves on pub.dev without my needing to be involved, but if it helps to get this thing off the ground, I don't mind having a more central effort.

By the way, if you are willing to keep working on it, I'd suggest opening up the GitHub issues page to invite collaborators to help submit bugs, etc. as well as get the license and copyright notice in place. Then I could also link from my project to your page and give it more attention. Either way is workable to me.

[GZGavinZhao]

@ryanheise Thanks for the quick help! Progress is unfortunately blocked because of issue flutter/flutter#52267, which basically says that there's no way of to creating a Dart implementation of a platform interface except for web. The problem is that if we use dart_vlc, we'll be definitely writing Dart code (and perhaps pure Dart code).

They did provide one way to manually force the use of pure Dart plugins (as shown in this line here taken from the first-party plugin path_provider). If we choose this work-around, our implementation will essentially become endorsed, which I'm not sure if this is something we want???

[ryanheise]

@GZGavinZhao Yes, I was coincidentally looking at path_provider last night for the same reason. So in the VLC case, something like this is now possible:

flutter:
  plugin:
    implements: just_audio
    platforms:
      linux:
        dartPluginClass: JustAudioVlcPlugin
        pluginClass: none

And then inside the Dart file we can have:

class JustAudioVlcPlugin extends JustAudioPlatform {
  final Map<String, JustAudioVlcPlayer> players = {};

  /// The entrypoint called by the generated plugin registrant.
  static void registerWith() {
    JustAudioPlatform.instance = JustAudioVlcPlugin();
  }

  ...
}

But no it won't become endorsed unless the main frontend plugin's pubspec file declares it so. Platform implementions can't endorse themselves.

By the way, it is still technically possible to get a Dart platform implementation working without the above feature, as I have used a workaround in just_audio_background where the app needs to manually invoke a static method to do the registration. In just_audio_background it makes sense because there are parameters we want to pass into init anyway.

[GZGavinZhao]

@ryanheise I tried to use this approach with the pubspec file but it didn't work. :( Here's what I currently have. Edit: use this one

In case you're wondering, the reason I added an extra "linux" after the plugin name and class is because I want to call some native code to do native volume controls and notifications, and this can help differentiate from the Windows implementation.

[ryanheise]

Just a quick comment, on the plugin name, but if the plugin is written entirely in Dart and you move all of the native code into a separate dependency, and if consequently the Windows and Linux Dart implementations are largely the same, you can also just write if/else statements depending on the platform.

[ryanheise]

@ryanheise I tried to use this approach with the pubspec file but it didn't work. :(

I just tried the path_provider example on linux and it works. I then tried un-endorsing the linux implementation and manually added it as a dependency to the example, and it still worked.

It's still a mystery to me what could be the difference between that and your project, but maybe it's a starting point for investigation?

[GZGavinZhao]

@ryanheise I'm gonna try this again tomorrow, but just to confirm, what branch & version of Flutter are you on?

[ryanheise]

I'm on stable. 2.2.2. (I've been prompted to upgrade, but I expect if this version works, newer versions will also work.)

[alexmercerind]

@ryanheise

Just a quick comment, on the plugin name, but if the plugin is written entirely in Dart and you move all of the native code into a separate dependency, and if consequently the Windows and Linux Dart implementations are largely the same, you can also just write if/else statements depending on the platform.

A package would be most likely what you must use. And use just_audio_vlc_linux as a dependency package.

@GZGavinZhao And, you can keep same package for both Windows & Linux (and remove _linux), since those two are the platforms that I'm supporting in my plugin.

This boilerplate can be removed after switching to package template, since there is no native implementation needed.

You might also wanna include // ignore_for_file: implementation_imports to import internal files from dart_vlc.

[GZGavinZhao]

@ryanheise For path_provider, they manually forced the use of their Linux implementation when the platform detected is Linux. See this line. So I believe for now, unless you manually put something similar here, like an if-else or a switch statement that chooses the implementation based on the platform, we're unable to do a pure Dart Linux implementation :(

[GZGavinZhao]

@ryanheise Okay I think my idea was on the right track. This was the manual override I was talking about, and the implementation will work after this.

@alexmercerind One little problem here: for a player's id, just_audio uses uuid while dart_vlc uses int... Will it be a big problem for you to change that? I can also try to crunch the uuid into a pointer (uuid is 128 bits and an unsigned Dart int is 64 bit).

[ryanheise]

For now, I would go for the solution used in just_audio_background so that the front-end plugin doesn't hard-code anything. What that involves is just exposing some static method that will install the platform implementation, and then any app that wants to use the Linux implementation should then manually call it in their main method:

if (!kIsWeb && Platform.isLinux) {
  JustAudioVlcPlugin.install();
}

Or alternatively you could expose a static method that does the conditional code itself:

void register() {
  if (!kIsWeb && Platform.isLinux) {
    _platform = JustAudioVlcPlugin();
  }
}

And from the app's main method call:

  JustAudioVlcPlugin.register();

This is similarly a temporary solution but it would just be a way of allowing multiple Linux implementations to exist.

[ryanheise]

@alexmercerind One little problem here: for a player's id, just_audio uses uuid while dart_vlc uses int... Will it be a big problem for you to change that? I can also try to crunch the uuid into a pointer (uuid is 128 bits and an unsigned Dart int is 64 bit).

Why can't you create a hash table that maps one ID onto the other?

[alexmercerind]

@alexmercerind One little problem here: for a player's id, just_audio uses uuid while dart_vlc uses int... Will it be a big problem for you to change that? I can also try to crunch the uuid into a pointer (uuid is 128 bits and an unsigned Dart int is 64 bit).

This infact is one of the reasons I assume that is bringing kinda confusion to everyone. I will try to add support for both int & String id for Player (from Dart itself). I didn't wanna use String as "identifier" because strings aren't a primitive data type in C & transferring them or any arrays through FFI is just bit of a work & I decided to avoid it as much as I could (not saying its impossible).

I believe going with what Ryan said would be great, possibly assign the id as the hash table's size for each new instance of Player. it will kinda work like an index.

[ryanheise]

Just FYI I have published a new audio_session which includes the no-ops for Linux and Windows.

[ryanheise]

Hi all. @bdlukaa has just published a just_audio_libwinmedia based on libwinmedia which adds Windows and Linux support to just_audio. The just_audio example directory has been updated to demonstrate how to use this, although it is currently untested for Linux. I was running into FFI issues when trying to build for Arch Linux, but YMMV. Please let me know below if anyone is able to get it working on their Linux distribution and we can try to improve the setup instructions.

[maks]

That looks very interesting, looking forward to trying it out on Linux.

Looks like the approach libwinmedia takes (at least on Linux) is to wrap the gtk webview to provide the play back functionality, certainly an interesting approach! I don't see that dependency documented in the libwinmedia readme so @ryanheise maybe that's what's causing your FFI issues?

[ryanheise]

I did have to manually install the shared lib dependencies, but even still I was getting a freeze without any actual error message. For me the freeze happens at the FFI call, and this never reaches the native plugin code. I've created an issue for libwinmedia here: https://github.com/harmonoid/libwinmedia/issues/6 .

[Sotneo]

@ryanheise @bdlukaa @alexmercerind I was able to run just_audio example without freezes under Linux after slightly patching libwinmedia(typo, merged) and example code(preload: false). Looks like something wrong with preloading.

      await _player.setAudioSource(
          AudioSource.uri(Uri.parse(
              "https://s3.amazonaws.com/scifri-episodes/scifri20181123-episode.mp3")),
          preload: false);

[bdlukaa]

Here is a list of features that are supported in comparison to windows:

Feature	Windows	Linux
read from URL	✅	✅
read from file	✅	✅
read from asset	✅	✅
read from byte stream
request headers
DASH	✅
HLS	✅
ICY metadata
buffer status/position	✅
play/pause/seek	✅	✅
set volume/speed	✅	✅
clip audio
playlists	✅	✅
looping/shuffling	✅
compose audio
gapless playback	✅
report player errors
handle phonecall interruptions
buffering/loading options
set pitch
skip silence
equalizer
volume boost

[ryanheise]

@ryanheise @bdlukaa @alexmercerind I was able to run just_audio example without freezes under Linux after slightly patching libwinmedia(typo, merged) and example code(preload: false). Looks like something wrong with preloading.

      await _player.setAudioSource(
          AudioSource.uri(Uri.parse(
              "http://10.217.251.154/calls/1516715759.19611-2018-01-23-17_55-109-89297076020.mp3")),
          preload: false);

Interesting. The preload parameter is actually handled completely in the Dart layer, and setting it to false basically delays the inevitable method call to the platform implementation to load the audio. Since the same load method is called in both cases, the only difference is "when" it is called. I would guess this is some sort of timing issue.

[bdlukaa]

It's probably because a webview is used in Linux, and it's loaded asynchronously.

[alexmercerind]

I implemented another bunch of missing features for Linux (I believe it is now same as Windows):

* PlayerAdd
* PlayerSetIsLooping
* PlayerSetIsAutoRepeat
* PlayerSetIsShuffling
* PlayerSetDownloadProgressEventHandler
* PlayerSetErrorEventHandler
* respective getters.

Crash was already resolved by @Sotneo the other day. I ask to give it another try (update just_audio_libwinmedia).

[alexmercerind]

That looks very interesting, looking forward to trying it out on Linux. Looks like the approach libwinmedia takes (at least on Linux) is to wrap the gtk webview to provide the play back functionality, certainly an interesting approach!

I'm glad to hear. :smile: I just decided to just wrap GTK WebKit webview, because GStreamer's pipeline based APIs were just wild. Other stuff like portaudio, ALSA, miniaudio were just too low level (I don't wanna implement network streaming from scratch) for the purpose, others were very strict licensed. This is providing a good set of features, APIs & good format support aswell (atleast all my music is playing). In case one wants more, they can just install gstreamer plugins from the distro package manager. Just only problem that took a lot of time to deal with was GTK being single threaded & web-view interop.

Off topic: In future, as we already have a platform view PR for Linux, I will be able to add video viewport too.

[ryanheise]

Awesome! Thanks, @Sotneo and @alexmercerind . It seems to be working well so far on my Linux box.

[Sotneo]

Awesome! Thanks, @Sotneo and @alexmercerind . It seems to be working well so far on my Linux box.

I am still experiencing freezes with examples. Some async staff inside _init() hangs the process. If I add some delay via await Future.delayed(const Duration(microseconds: 1));. All examples are working.

[bdlukaa]

Hello everyone! I will publish the new version of just_audio_libwinmedia with libwinmedia updated

[defsub]

I'm also seeing the mentioned freezing both with local and remote URLs using my own app on Linux. I noticed that request headers aren't supported yet (which I need) so that may be one issue, but local files also freeze. I'm happy to do further testing with the next update.

[nathanfranke]

Edit: if you are having an error (#44) using just_audio on Linux, install the just_audio_libwinmedia package.

See also: https://github.com/ryanheise/just_audio/issues/582