search

eshaz / icecast-metadata-js

Browser and NodeJS packages for playing and reading Icecast compatible streaming audio with realtime metadata updates.
158 stars 20 forks source link

Icecast only loading or no sound on iOS in Angular #119

Closed wunschradio closed 2 years ago

wunschradio commented 2 years ago

Hello Ethan,

first of all thank you for your hard work. I really like it and it works perfectly on Android. Unfortunately I can't get it to work on iOS. I am using Ionic and Angular. The stream is loading (network tab of developer console), but I can't hear anything. The event listener gives success on play, no error. The stream keeps on loading.

If I use a standard html5 audio without your script the stream plays fine. I really look forward to use your icecast-metadata-player.

Do you have any idea?

eshaz commented 2 years ago

Which version of iOS are you testing with? Also, could you try out the demo page of this repo on the iOS device you're testing with and see if that works?

You might want to look through some of the previous issues here regarding iOS, if you haven't already. There might be some information in there you would find helpful.

wunschradio commented 2 years ago

Thank you for your reply. Using iOS 15.4 and latest cordova-ios@6.2. Tested with older cordova-ios versions, too. The demo page works in the browser of the phone. I read all I could find about it. Been struggeling with it a couple of days now.

eshaz commented 2 years ago

I'm not familiar with Cordova.

I would suggest putting together a test app using that tool with just a play button and a hardcoded stream URL and see if that works. Essentially to try narrowing down the issue if one exists in the library

There are also a few parameters you might for IcecastMetadataPlayer that you could try such as changing the playbackMethod and see if you get any more feedback from that.

wunschradio commented 2 years ago

Unfortunately the same. I build a very minimalistic app. Stream loads, no play. As it is working in browser I think its because of my setup with Cordova. Maybe somewhen somebody might post a solution here.

jonathandv commented 2 years ago

I have an icecast mp3 stream that does play with IcecastMetadataPlayer on iOS Safari and on iOS Firefox. I can't hear anything on iOS Chrome though. And I can only visualize the audio (as bars that go up and down in a frequency spectrum, using AnalyserNode) in iOS if I use an html audio tag only, so without IcecastMetadataPlayer. In desktop Chrome and Android Chrome the audio does play and I can also visualize the audio (with IcecastMetadataPlayer and AnalyserNode). On iOS I just use a generic animation that does not reflect the audio levels.

My goal at the moment is to hear sound in iOS Chrome. I might be able to provide useful debug information.

eshaz commented 2 years ago

That's very odd that there is no sound in only iOS Chrome, but other iOS browsers work fine. As far as I'm aware, iOS uses the same WebKit engine behind the scenes for all browsers.

What version of iOS and Chrome are you testing with?

In iOS, MP3 and Opus streams are decoded in WASM and played through the WebAudio API. If you want to clone down the repo and debug, you'd probably have the best luck starting here where the AudioContext is set up before playback starts.

https://github.com/eshaz/icecast-metadata-js/blob/5ffd900fe5e18fe8425e0ab4ac7b5efad5fe816c/src/icecast-metadata-player/src/players/WebAudioPlayer.js#L83

jonathandv commented 2 years ago

Thanks for your reply!

I'm testing on an iPad with iOS 15.3.1 and iOS Chrome 101.0.4951.58. I've noticed that with iOS Chrome I also can't hear sound in the demo on https://eshaz.github.io/icecast-metadata-js/ while iOS Safari does produce sound.

I've set up my environment such that I can put breakpoints in iOS Safari and view the console of iOS Chrome (unfortunately I don't know how I can arrange breakpoints in iOS Chrome). I've added console.log()'s to IcecastMetadataPlayer though, and I can already see a difference in the logs between my app in iOS Safari and iOS Chrome.

iOS Safari: IcecastMetadataPlayer.play() gets called twice, and the sound works. iOS Chrome: IcecastMetadataPlayer.play() gets called once, and then WebAudioPlayer._getAudioContext() gets called once, and the sound doesn't work.

WebAudioPlayer._getAudioContext() does not get called in iOS Safari.

There is a chance that this difference is related to my implementation in my app, but I don't think I'm doing anything special. I've just followed the example from the documentation.

I suppose now we just have to figure out where the code paths start to deviate for the 2 browsers. I'm not very familiar with IcecastMetadataPlayer but I'm willing to investigate further :) Any tips though are greatly appreciated!

Update: iOS Chrome throws an error "Can't find variable: MediaSource" when calling "new MediaSource()". From there on a different player is tried. I'm not sure if we have to make the MediaSource work (if possible, it's experimental) or make WebAudio work. According to caniuse.com MediaSource should be fully supported by the versions of Safari and Chrome that I'm testing on, but no luck yet with Chrome.

eshaz commented 2 years ago

Would you be able to paste a stack trace that shows where the MediaSource error is being thrown? I have a guess where that's coming from, but I'm not entirely sure.

Could you also try setting playbackMethod to webaudio and html5 in the options when instantiating IcecastMetadataPlayer? You can also select the playback method in the HTML demo page.

Worst case scenario, you should still get playback through the HTML5 audio element playback method. There are a few things that can prevent HTML5 audio playback, other than a bug in the library, such as not interacting with the page to start playback: see https://github.com/eshaz/icecast-metadata-js/issues/107#issue-1154526730. Sometimes there are quirks / differences in how browser enforce standards like this.

MediaSource support has been absent in iOS for a very long time even though it's been a standard for many years now. (Probably not due to any technical reasons...) Now that you mention you're using an iPad, I remember reading somewhere that MediaSource support was added to some degree on the iPad. I've never tried MediaSource on an iPad, since I can't install Chrome on the iPad Simulator. It's possible there are quirks in how it's implemented.

Some background on how IcecastMetadataPlayer works: IcecastMetadataPlayer tries to detect the supported playback method by reading the audio codec from the Icecast stream and then checking for MediaSource, WebAudio, and HTML5 support. If MediaSource and Web Audio aren't supported by the browser or the audio codec isn't supported, then HTML5 audio is used as a fallback. ICY and Ogg Metadata will still be read and displayed by the first request, and a second request is started to play the audio in the HTML5 Audio element. This isn't ideal, since there are two requests, one for metadata and another for audio playback, but it shouldn't really happen since MediaSource and WebAudio should be supported by all mainstream browsers.

jonathandv commented 2 years ago

Thanks for the explanation! I've also read the code and I understand the flow now.

Here is the stack trace for the thrown error in iOS Chrome:

ERROR ReferenceError: Can't find variable: MediaSource
canPlayType        @http://**************/main.cd8c52d5d43dfba6.js:1:280478
_buildPlayer       @http://**************/main.cd8c52d5d43dfba6.js:424:411410
                   @http://**************/main.cd8c52d5d43dfba6.js:424:411049
generatorResume    @[native code]
Ox                 @http://**************/main.cd8c52d5d43dfba6.js:1:218915
o                  @http://**************/main.cd8c52d5d43dfba6.js:1:219120
onInvoke           @http://**************/main.cd8c52d5d43dfba6.js:1:99766
run                @http://**************/polyfills.9e082b696a4b45bb.js:1:1952
                   @http://**************/polyfills.9e082b696a4b45bb.js:1:16785
onInvokeTask       @http://**************/main.cd8c52d5d43dfba6.js:1:99586
runTask            @http://**************/polyfills.9e082b696a4b45bb.js:1:2573
_                  @http://**************/polyfills.9e082b696a4b45bb.js:1:9190
promiseReactionJob @[native code]

This is in WebAudioPlayer.js on line 43:

new MediaSource();

Even though caniuse.com says MediaSource is supported by iOS Chrome (since iOS 13, I think), it's not working. In iOS Safari "new MediaSource();" DOES work and it is used as player. I've also tried WebKitMediaSource but it cannot be found by iOS Chrome either.

Assuming that iOS Chrome just doesn't work with MediaSource, I can try and get WebAudio to work. It seems possible because for at least 1 particular mp3 stream on the HTML Demo page, iOS Chrome DOES play sound (though not in the React app...). The HTML demo tells me that WebAudio is being used in that case.

(Setting HTML5 as player in my app does not work. No sound played, but I can try debug it later.)

Now, I'm trying WebAudio and adding console.logs to WebAudioPlayer._getAudioContext(). I can see that the audio context state is set to 'running' when resume() is called. But no sound yet. Any idea which property is good to inspect? I can't compare to iOS Safari because setting WebAudio as playback method gives me a different error in that browser... (which is ok, because i can revert to MediaSource in iOS Safari).

eshaz commented 2 years ago

Thanks for the stack trace and the debugging so far. I fixed that MediaSource error and a bug in iOS playback #121.

There was a bug when using the Audio element controls in iOS where the AudioContext wouldn't start properly so there would be no sound. iOS prevents AudioContext from running when it's not triggered directly by a user interaction, even when the user clicks the "play" button on the Audio element controls. I'm not sure if this was related to your issue.

Could you give icecast-metadata-player/1.11.7 a try and see if that helps or changes anything?

jonathandv commented 2 years ago

I've tried version 1.11.7 but I can't hear sound yet in iOS Chrome. Looking at the changes in #121 I assume an audio element needs to be present in the html? I did not have that before. So I've now tried adding it and passing it on to the IcecastMetadataPlayer constructor but no luck in iOS Chrome. I do see in my logs that when I click on the play button of the audio element, the resume() promise in unlock() is being called and resolved. As an experiment I also tried removing the call to resume() in WebAudioPlayer._initAudioContext(), because it seems to me it shouldn't be used together with the other resume()?

Unfortunately I don't know the ins and outs of the code and of playing audio on the web well enough to know what to do. But I can help with testing and experimenting when needed. Sound does play on iOS Safari, which I assume most iOS users will use, but it would be great to support iOS Chrome as well. I still have hope that it can work, because the HTML demo page DOES play sound in iOS Chrome for some of the streams I tried.

I appreciate your help so far. Let me know if I can be of further assistance. A clear head after taking a break might also help me :)

wunschradio commented 2 years ago

Same here. I was hoping this would fix my problem, too. The stream is loading but I can't hear any sound. The log looks the same as on android, no errors - load, start buffer and play is triggered.

eshaz commented 2 years ago

@jonathandv Which streams work and don't work on the demo page? If there's some correlation to audio codec or something like that, it would help narrow down this issue. Could you also post a screenshot of the supported codecs table on the HTML Demo page that shows up on iOS Chrome? It's the table just below the player controls on the demo. What audio codec is the stream that you have on your page?

The Audio element isn't required to be in the HTML, it's just one of the ways to use IcecastMetadataPlayer. This is useful if someone just wants to use the controls on the Audio element to start and stop playback, rather than using the .play() and .stop() methods. Essentially this makes it a drop in library for a page that already has an Audio element on it.

In the meantime, I'm going to attempt installing Chromium on the iOS simulator. If I'm successful, I should be able to reproduce the issue you are seeing.

jonathandv commented 2 years ago

With latest versions of iOS Chrome and iOS Safari.

Stream: https://dsmrad.io/stream/isics-all iOS Chrome, HTML demo : ok iOS Chrome, React demo: sound does not work (but does show stream metadata). iOS Safari, HTML demo : ok iOS Safari, React demo: ok

Stream: https://ice2.somafm.com/groovesalad-256-mp3 iOS Chrome, HTML demo : ok iOS Chrome, React demo: sound does not work (says: "Waiting for metadata..."). iOS Safari, HTML demo : ok iOS Safari, React demo: ok

Streams other than mp3 don't seem to play in iOS Chrome in the HTML demo (at least: no audio).

My stream I prefer not to share right now (not meant for public), but it is an icecast mp3 stream and I printed the codec to the console: 'audio/mpeg; codecs="mpeg"'. It plays in iOS Safari using MediaSource, and does not play in iOS Chrome (using WebAudio). But I've been testing with the 2 public streams above, to make the scenario as close as possible in my app.

iOS Safari: iOS_Safari

iOS Chrome: iOS_Chrome

eshaz commented 2 years ago

I think I finally found the issue on this one.

One of my downstream dependencies had a compatibility issue with bundlers that polyfill ES6 classes. This was causing the web audio decoder to not serialize all of the code when it is loaded as a web worker. In the React demo, this error only happens in the production build, which is heavily minified and polyfilled by the create-react-app build pipeline. Additionally, it only shows up on platforms that don't support MediaSource, i.e. iOS.

I've released a fix in icecast-metadata-player/1.11.8. Could you try that out?

jonathandv commented 2 years ago

Untitled

Thank you so much! :) The sound now also works properly in iOS Chrome. I've tested your React demo app and my own app.

I'll have a look at the changes and learn from them. Getting to know the code better will make it easier to contribute, maybe the next time.

I did notice 2 things:

eshaz commented 2 years ago

@wunschradio, have you had a chance to try out icecast-metadata-player/1.11.9 to see if that solves your issue?

wunschradio commented 2 years ago

@eshaz YES! Thank you!

eshaz commented 2 years ago

@eshaz YES! Thank you!

@wunschradio Excellent, thanks for confirming!

when I stop the sound and try to start it again it doesn't play anymore (only in iOS Chrome, in my app).

@jonathandv If you think this is a problem with this library, could you open another issue? We can continue investigating there if needed.

Thank you both for helping work through this issue!