Web support

[nottheswimmer]

Is your feature request related to a problem? Please describe. I would like audio_service to work on the web. It appears that, for my application at least, it does not.

Describe the solution you'd like Add web support to audio_service.

Describe alternatives you've considered Implementing a custom fallback.

Additional context My application using audio service that is having problems: https://github.com/nottheswimmer/podcastsync/commit/33f8f96415d201b180e4ed82bde2b15b50c8bb37

You can see it failing here: https://podcastsync.michaelphelps2.repl.co/

The first attempt (starting the audio service) yields...

errors.dart:146 Uncaught (in promise) Error: UnimplementedError
    at Object.throw_ [as throw] (errors.dart:195)
    at Function.getCallbackHandle (window.dart:1067)
    at start (audio_service.dart:568)
    at start.next (<anonymous>)
    at runBody (async_patch.dart:86)
    at Object._async [as async] (async_patch.dart:125)
    at Function.start (audio_service.dart:549)
    at _handlePlayerEvent (navigation-bloc.dart:41)
    at _handlePlayerEvent.next (<anonymous>)
    at runBody (async_patch.dart:86)
    at Object._async [as async] (async_patch.dart:125)
    at navigation_bloc.NavigationBloc.new.[_handlePlayerEvent] (navigation-bloc.dart:38)
    at _RootZone.runUnaryGuarded (zone.dart:1374)
    at _BroadcastSubscription.new.[_sendData] (stream_impl.dart:339)
    at _DelayedData.new.perform (stream_impl.dart:594)
    at _StreamImplEvents.new.handleNext (stream_impl.dart:710)
    at async._AsyncCallbackEntry.new.callback (stream_impl.dart:670)
    at Object._microtaskLoop (schedule_microtask.dart:43)
    at _startMicrotaskLoop (schedule_microtask.dart:52)
    at async_patch.dart:168

Subsequent attempts yield...

html_dart2js.dart:39777 Overflow on channel: ryanheise.com/audioService.  Messages on this channel are being discarded in FIFO fashion.  The engine may not be running or you need to adjust the buffer size if of the channel.
error @ html_dart2js.dart:39777
_printDebugString @ natives.dart:23
_printDebug @ natives.dart:15
push @ channel_buffers.dart:147
handlePlatformMessage @ plugin_registry.dart:82
runBody @ async_patch.dart:86
_async @ async_patch.dart:125
handlePlatformMessage @ plugin_registry.dart:76
[_sendPlatformMessage] @ window.dart:450
sendPlatformMessage @ window.dart:318
[_sendPlatformMessage] @ binding.dart:167
send @ binding.dart:223
_invokeMethod @ platform_channel.dart:146
runBody @ async_patch.dart:86
_async @ async_patch.dart:125
[_invokeMethod] @ platform_channel.dart:144
invokeMethod @ platform_channel.dart:329
addQueueItem @ audio_service.dart:616
runBody @ async_patch.dart:86
_async @ async_patch.dart:125
addQueueItem @ audio_service.dart:616
_handlePlayerEvent @ navigation-bloc.dart:48
runBody @ async_patch.dart:86
_async @ async_patch.dart:125
[_handlePlayerEvent] @ navigation-bloc.dart:38
runUnaryGuarded @ zone.dart:1374
[_sendData] @ stream_impl.dart:339
perform @ stream_impl.dart:594
handleNext @ stream_impl.dart:710
(anonymous) @ stream_impl.dart:670
_microtaskLoop @ schedule_microtask.dart:43
_startMicrotaskLoop @ schedule_microtask.dart:52
(anonymous) @ async_patch.dart:168
Promise.then (async)
_scheduleImmediateWithPromise @ async_patch.dart:169
_scheduleImmediate @ async_patch.dart:138
_scheduleAsyncCallback @ schedule_microtask.dart:73
_rootScheduleMicrotask @ zone.dart:1245
scheduleMicrotask @ schedule_microtask.dart:139
[_tryToResolveArena] @ arena.dart:230
[_resolve] @ arena.dart:214
resolve @ arena.dart:51
resolve @ recognizer.dart:259
didStopTrackingLastPointer @ monodrag.dart:340
stopTrackingPointer @ recognizer.dart:340
[_giveUpPointer] @ monodrag.dart:356
handleEvent @ monodrag.dart:280
[_dispatch] @ pointer_router.dart:75
(anonymous) @ pointer_router.dart:121
forEach @ linked_hash_map.dart:23
[_dispatchEventToRoutes] @ pointer_router.dart:118
route @ pointer_router.dart:105
handleEvent @ binding.dart:217
dispatchEvent @ binding.dart:196
[_handlePointerEvent] @ binding.dart:155
[_flushPointerEventQueue] @ binding.dart:101
[_handlePointerDataPacket] @ binding.dart:85
_invoke1 @ window.dart:590
invokeOnPointerDataPacket @ window.dart:238
[_onPointerData] @ pointer_binding.dart:129
(anonymous) @ pointer_binding.dart:457
(anonymous) @ pointer_binding.dart:418
(anonymous) @ pointer_binding.dart:195
Show 3 more frames
errors.dart:146 Uncaught (in promise) Error: MissingPluginException(No implementation found for method connect on channel ryanheise.com/audioService)
    at Object.throw_ [as throw] (errors.dart:195)
    at MethodChannel._invokeMethod (platform_channel.dart:154)
    at _invokeMethod.next (<anonymous>)
    at onValue (async_patch.dart:47)
    at _RootZone.runUnary (zone.dart:1439)
    at _FutureListener.thenAwait.handleValue (future_impl.dart:141)
    at handleValueCallback (future_impl.dart:682)
    at Function._propagateToListeners (future_impl.dart:711)
    at _Future.new.[_completeWithValue] (future_impl.dart:526)
    at async._AsyncCallbackEntry.new.callback (future_impl.dart:556)
    at Object._microtaskLoop (schedule_microtask.dart:43)
    at _startMicrotaskLoop (schedule_microtask.dart:52)
    at async_patch.dart:168

[ryanheise]

This plugin doesn't make much sense in the context of the web, since it's two main goals are:

1. Provide a context to play audio in the background, while the screen is turned off.
2. Provide access to remote control interfaces, such as your phone's lock screen or headset.

Or are you just proposing a trivial implementation that merely provides the same API, but doesn't actually do the 2 primary functions listed above?

[nottheswimmer]

At the very least, the audio service shouldn't prevent the underlying audio from playing (my app should still work as a podcast player even on the web, even if the background audio service has to be disabled).

However, I think this still applies to web applications for mobile devices. I would like my app to behave similar to soundcloud and still play in the background from my phone. Here is an example of this:

[ryanheise]

I'll leave this issue open to collect thoughts on how this plugin could be interpreted in a web context.

I am not familiar with the background audio capabilities of web apps on mobile phones, but from that screenshot, it appears some level of support is possible.

[nottheswimmer]

Doing a bit of research on my own. I'm having trouble finding the right way to search for information about this particular problem... Nonetheless, I've created the simplest possible audio player by referencing this page from the Mozilla Foundation.

Here is a link: https://defaultwebaudioplayer.michaelphelps2.repl.co/

By default, on my Android Phone running Chrome, it looks like this (it has a functioning pause/play button but not much else):

Somehow soundcloud was able to...

• theme it
• give it a title
• give it forward and backwards buttons

It also, on soundcloud, if expanded shows the length of a video and a seeker bar -- although the time is never actually updated and the bar doesn't do anything.

-- MERGED WITH DELETED POST BELOW --

https://developers.google.com/web/updates/2017/02/media-session https://developer.mozilla.org/en-US/docs/Web/API/Media_Session_API

Aha, I think I finally found some useful documentation... Also, somebody at Google thinks they're funny.

EDIT: So here it is then... https://repl.it/@MichaelPhelps2/CustomizedMediaSession

Note that none of these extra controls actually work presently (there are functions exposed that define what happens when the events get triggered -- hopefully that's as simple as mapping them to one of the existing functions you have in audio_service) but, it's a fine proof of concept.

EDIT 2: Some very nice documentation on what platforms are supported and what it looks like on various platforms (last updated Aug 20, 2019) https://github.com/w3c/mediasession/blob/master/explainer.md

[nottheswimmer]

Honestly, just providing a way for the audio to pass through on unsupported platforms would be a huge deal for me. I've tried rewriting my code to call these audio functions directly when kIsWeb is true, but the only way I'm seeing is to completely fork my widget.

[ryanheise]

@nottheswimmer that's a convincing point. I agree that even if your app doesn't need to use any of the features this plugin was designed for, making it at least implement the pass-throughs will make it easier to port apps to the web platform. That would at least be a good first step.

[dxvid-pts]

I want to add web support to one of my projects, and this plugin is a blocker for web migration, as it would require me to rewrite all of the audio logic.

To add a bit of input to the conversation above: desktop browsers show media notifications as well. This is not a mobile only issue.

P.S. this would also increase the plugin score on pub.dev

[ryanheise]

Hi @peterscodee I agree with you 100%. Would you be interested in helping/contributing?

Ideally I alone would be able to implement all features in a timely manner, but I have a long backlog, and I tend to put my efforts into the changes that involve large architectural decisions (currently #415 ), and other people have graciously contributed pull requests for other features that are important to them. If you do decide to have a crack at it, your contributions will be most welcome.

Otherwise, it is a matter of when I'll get around to it (note that it took me almost 2 years to get around to #415 !)

[hacker1024]

I'd image web support won't be too difficult, but we might need to abstract the existing isolate code and split it up in a way that would allow either a background isolate to be launched and interacted with (for the existing platforms), or use an alternative web implementation that provides the same APIs.

That's a pretty major architectural change, and I fear that it may be hard to implement and merge in while the master branch continues to change.

[ryanheise]

A cheap first solution would do as little as possible to just pass all AudioService methods through to the ryanheise.com/audioServiceBackground method channel, without actually providing a media notification of any sort, and without emulating the isolation provided by the Android and iOS implementations. That would at least allow a project to compile and run on web. (The issue of isolates on the web is tricky, so we can probably not bother to solve that side of things perfectly until Flutter/Dart comes out with a proper solution that we can use.)

That's a pretty major architectural change, and I fear that it may be hard to implement and merge in while the master branch continues to change.

I don't think the web implementation needs to involve any major architectural change. It can be implemented in a very similar way to the iOS implementation but without actually having a separate isolate.

My personal priorities are to work on the major architectural changes, of which #415 is the current main one, but if someone does manage to contribute a web implementation before then, I would be happy to do the work of merging that into #415 . I should say that I don't expect #415 to land any time soon. It could take well over a month for me to get this working.

[dxvid-pts]

Hi @peterscodee I agree with you 100%. Would you be interested in helping/contributing?

I looked into the code and I don't think that I am able to migrate to web. I'll try but I can't promise anything. However, I investigated a bit and it seems that the Media Session API is the standard across browsers. -> https://w3c.github.io/mediasession/

An example can be found here: https://googlechrome.github.io/samples/media-session/

It would be great if somebody can help to migrate to web!

[ryanheise]

Big thanks to @keaganhilliard we now have a pull request for this which is now merged. Please try it out on git master.

[dxvid-pts]

That's so cool! I've tried it and so far I've noticed two things. Notification images and custom buttons via MediaControl aren't working. But great work 👍

[keaganhilliard]

Thanks for checking it out! Yeah the image is a weird thing with the service worker trying to cache and causing a CORS error. I think I have a workaround I'm testing out. As far as custom actions go, I haven't found anything that says the mediaSession supports them. I have seen one in the wild, but I have no idea how they managed it (YouTube Music).

[dxvid-pts]

As far as custom actions go, I haven't found anything that says the mediaSession supports them. I have seen one in the wild, but I have no idea how they managed it (YouTube Music).

I think it's done with setActionHandler https://developer.mozilla.org/en-US/docs/Web/API/MediaSession/setActionHandler I'll take a look tomorrow.

Edit: You already used setActionHandler. My guess was wrong.

[keaganhilliard]

Yeah setActionHandler expects a specific set of valid strings, all of which I have been able to implement except seekto. I am working on a workaround for that as well 😄.

Edit: Also didn't implement skipad

[keaganhilliard]

@peterscodee The artwork and seekTo should be working now. I did some more research and it doesn't look like custom actions are supported at all. The one I mentioned before is Picture in picture which is a chrome feature for videos. I could potentially expose skipad for a single custom action, but other than that I think we have support for everything that the MediaSession api provides.

[ryanheise]

@keaganhilliard I've just done some refactoring and updated the feature list in the README - hopefully I didn't break anything! Let me know if I have.

[ryanheise]

This has now been released in 0.15.0 . Thanks, @keaganhilliard for the great contribution. I'll close this issue and any future issues related to web support can be created as new issues.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs, or use StackOverflow if you need help with audio_service.