When a stream gets interrupted "playing silence to avoid xrun" is not a solution

[GK-774]

If, for a stream, a network interruption occurs which exceeds the buffer, silence to avoid xrun is played but obviously no reconnect is attempted. This will go on forever if not manually interrupted. A stop and start reconnected to the stream immediately in all cases I have encountered so far.

e.g: Sep 27 19:38:01 mpd8.hal9000.at mpd[328913]: alsa_output: Decoder is too slow; playing silence to avoid xrun ... and so on Sep 27 19:47:41 mpd8.hal9000.at mpd[328913]: alsa_output: Decoder is too slow; playing silence to avoid xrun

In case of a stream just waiting for data to come in again, like in case of a static file, is obviously no solution, it won´t come in. The streaming server expects the client to reconnect (or it´s gone). The default buffer time for (as an example) shoutcast server is 1s, so it would be sensible that after 1s of playing silence a reconnect (or a stop / start sequence) is attempted.

[GK-774]

Bug report

Describe the bug

If a http stream is interrupted for a short time, wheter a reconnect is attempted nor a stop of playback is done. Instead silence is played endlessly. A mpc stop and mpc start immidiately reconnects to the stream and plays normally then.

Expected Behavior

Reconnecting to the stream would be attempted (basically stop/start) or if this is not feasible to implement, playback stopped and an error reported.

Actual Behavior

The actual error condition is obfuscated by playing silence in an endless loop.

Version

Music Player Daemon 0.23.9 (0.23.9) Copyright 2003-2007 Warren Dukes warren.dukes@gmail.com Copyright 2008-2021 Max Kellermann max.kellermann@gmail.com This is free software; see the source for copying conditions. There is NO warranty; not even MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Database plugins: simple proxy upnp

Storage plugins: local udisks nfs curl

Neighbor plugins: upnp udisks

Decoders plugins: [mad] mp3 mp2 [mpg123] mp3 [vorbis] ogg oga [oggflac] ogg oga [flac] flac [opus] opus ogg oga [sndfile] wav aiff aif au snd paf iff svx sf voc w64 pvf xi htk caf sd2 [audiofile] wav au aiff aif [dsdiff] dff [dsf] dsf [hybrid_dsd] m4a [faad] aac [wavpack] wv [modplug] 669 amf ams dbm dfm dsm far it med mdl mod mtm mt2 okt s3m stm ult umx xm [ffmpeg] 16sv 3g2 3gp 4xm 8svx aa3 aac ac3 adx afc aif aifc aiff al alaw amr anim apc ape asf atrac au aud avi avm2 avs bap bfi c93 cak cin cmv cpk daud dct divx dts dv dvd dxa eac3 film flac flc fli fll flx flv g726 gsm gxf iss m1v m2v m2t m2ts m4a m4b m4v mad mj2 mjpeg mjpg mka mkv mlp mm mmf mov mp+ mp1 mp2 mp3 mp4 mpc mpeg mpg mpga mpp mpu mve mvi mxf nc nsv nut nuv oga ogm ogv ogx oma ogg omg opus psp pva qcp qt r3d ra ram rl2 rm rmvb roq rpl rvc shn smk snd sol son spx str swf tak tgi tgq tgv thp ts tsp tta xa xvid uv uv2 vb vid vob voc vp6 vmd wav webm wma wmv wsaud wsvga wv wve rtp:// rtsp:// rtsps:// [pcm]

Filters: libsamplerate soxr

Tag plugins: id3tag

Output plugins: shout null fifo pipe alsa ao oss openal httpd snapcast recorder

Encoder plugins: null vorbis opus lame twolame wave flac

Archive plugins: [bz2] bz2 [zzip] zip [iso] iso

Input plugins: file archive alsa qobuz curl ffmpeg nfs mms cdio_paranoia

Playlist plugins: extm3u m3u pls xspf asx rss soundcloud flac cue embcue

Protocols: file:// alsa:// cdda:// ftp:// ftps:// gopher:// hls+http:// hls+https:// http:// https:// mms:// mmsh:// mmst:// mmsu:// nfs:// qobuz:// rtmp:// rtmpe:// rtmps:// rtmpt:// rtmpte:// rtp:// rtsp:// rtsps:// scp:// sftp:// smb:// srtp://

Other features: dbus udisks epoll icu inotify ipv6 systemd tcp un

Configuration

music_directory "/var/lib/mpd/music" playlist_directory "/var/lib/mpd/playlists" state_file "/var/lib/mpd/mpdstate" sticker_file "/var/lib/mpd/sticker.sql"

log_level "notice"

database { plugin "simple" path "/var/lib/mpd/mpd.db" cache_directory "/var/lib/mpd/cache" }

user "mpd"

restore_paused "yes" auto_update "no" replaygain "off"

input { plugin "curl" }

audio_output { type "alsa" name "Topping D30pro" device "hw:Pro,0" mixer_type "none" auto_resample "no" auto_format "no" auto_channels "no" replay_gain_handler "none" }

resampler { plugin "libsamplerate" type "1" }

input { enabled "no" plugin "qobuz" }

Log

... Sep 25 16:54:33 mpd8.hal9000.at mpd[211678]: alsa_output: Decoder is too slow; playing silence to avoid xrun and so on until stop/start ... Sep 26 12:18:02 mpd8.hal9000.at mpd[211678]: alsa_output: Decoder is too slow; playing silence to avoid xrun and so on until stop/start ... Sep 27 19:38:01 mpd8.hal9000.at mpd[328913]: alsa_output: Decoder is too slow; playing silence to avoid xrun and so on until stop/start ... Sep 28 17:17:07 mpd8.hal9000.at mpd[402564]: alsa_output: Decoder is too slow; playing silence to avoid xrun and so on until stop/start ... Sep 28 17:33:58 mpd8.hal9000.at mpd[402564]: alsa_output: Decoder is too slow; playing silence to avoid xrun and so on until stop/start ... Sep 30 07:42:28 mpd8.hal9000.at mpd[406162]: alsa_output: Decoder is too slow; playing silence to avoid xrun Sep 30 07:42:33 mpd8.hal9000.at mpd[406162]: alsa_output: Decoder is too slow; playing silence to avoid xrun Sep 30 07:42:38 mpd8.hal9000.at mpd[406162]: alsa_output: Decoder is too slow; playing silence to avoid xrun Sep 30 07:42:43 mpd8.hal9000.at mpd[406162]: alsa_output: Decoder is too slow; playing silence to avoid xrun Sep 30 07:42:48 mpd8.hal9000.at mpd[406162]: alsa_output: Decoder is too slow; playing silence to avoid xrun Sep 30 07:42:53 mpd8.hal9000.at mpd[406162]: alsa_output: Decoder is too slow; playing silence to avoid xrun Sep 30 07:42:55 mpd8.hal9000.at mpd[406162]: player: played "http://orf-live.ors-shoutcast.at/oe1-q2a"

[MaxKellermann]

Under what exact circumstances do you wish a change of MPD's behavior? Your description is a bit vague.

[GK-774]

As it is now, when a http stream experiences a short interruption (e.g. network glitches) mpd starts to "play silence" and never stops this (can go on forever) until it´s manually stopped. So either mpd should try to reconnect (and switch from silence to playing the actual stream) or if that is not doable, it should stop. Ideally if a http stream gets interrupted mpd would try to reconnect after a second or so (e.g. by stop/start) and probably retry e.g. 3 times. If the stream is still unavailable the stop and throw a message.

[GK-774]

This describes a similar issue with a long interruption (hibernate), but how the reconnect should be handled is the same: https://github.com/MusicPlayerDaemon/MPD/issues/361 So similar expected behaviour: MPD tries to reconnect optional: MPD sends a short notice to clients eg reconnecting to stream server MPD reconnects to streaming source. If connect fails MPD continues with next item on playlist (or stops)

[MaxKellermann]

MPD never stops because the TCP connection to the server remains open. As soon as your operating system's TCP stack observes that the connection is gone, MPD will stop playback (and go to the next song in the queue).

So either mpd should try to reconnect (and switch from silence to playing the actual stream) or if that is not doable, it should stop.

Does that mean you wish MPD to reconnect or stop as soon as the first xrun occurs?

[GK-774]

TCP connection to the server remains open - does it? I rather think the streaming server on the other end drops the connection to have not tons of dead sessions for clients that might or might not come back. If I do a start and stop (and thus reconnecting) the stream plays immidiately, if I don´t there is playing silence indefinitely. And yes, reconnecting to the stream would be ideal (basically the same thing as I´m doing manually with stop/start now. For a live stream we could not continue where we stopped anyway but have to jump in at the current position.

[MaxKellermann]

TCP connection to the server remains open - does it?

How can I know? It's your computer.

And yes, reconnecting to the stream would be ideal

Under what exact circumstances? You didn't answer that question.

[GK-774]

Under what exact circumstances? - This would be a possible scenario:

1) Source is a stream (curl plugin) 2) The alsa buffer gets empty -> the stream was interrupted for whatever reason 3) MPD initiates a reconnect (while playing silence) 4) If unsucessful, 2nd reconnection attempt after 1s 5) If still unsucessful, 2nd reconnection attempt 2s later 6) If still unsucessful, 2nd reconnection attempt 4s later 7) If still unsucessful, 2nd reconnection attempt 8s later 8) If still unsucessful, (now 15s have elapsed) declare the stream dead 9) Move on to the next item in the queue or stop.

Such interruptions are not very frequent (once a day on average) and short, but they do happen. Typically we would be reconnected after 1s or so.

[GK-774]

Correction: 5) 3rd, 6) 4th, 7) 5th, a total of 5 reconnection attempts worst case.

[MaxKellermann]

2. The alsa buffer gets empty -> the stream was interrupted for whatever reason

This is a "yes" to my question you didn't answer yet. This means every time there's some clock jitter or a tiny network (wifi) hiccup or packet loss, MPD will reconnect, causing a delay of several seconds until it can resume playback, instead of just a very short click. That's not good, that's not what you want.

[MaxKellermann]

If unsucessful, 2nd reconnection attempt after 1s

And this: how does an "unsuccesful reconnect" look like, exactly? You said attempt to reconnect after 1s - but how long does the first reconnect take? Count 1s from which point in time - from starting the first reconnect, or from giving up the first reconnect? When shall MPD give up the reconnect?

You see, your problem gets more complex the deeper you look, even though it sounded so easy when you didn't know all the details. Right now, MPD uses your operating system's default and relies that your operating system is configured well. But if you want to override those defaults, you need a full-blown concept which considers all the small details, because you need to reimplement everything that usually your operating system (and its TCP stack) do.

[MaxKellermann]

If still unsucessful, (now 15s have elapsed) declare the stream dead

And this: 15s have elapsed since when? Since the first buffer xrun? One single connection attempt can take a minute or so - that's how long it can take to establish a TCP connection over the internet to a server on the other side of the globe. Just the first reconnect attempt can take longer than your total of 15s.

[GK-774]

"This is a "yes" to my question you didn't answer yet. This means every time there's some clock jitter or a tiny network (wifi) hiccup or packet loss, MPD will reconnect, causing a delay of several seconds until it can resume playback, instead of just a very short click. That's not good, that's not what you want."

"instead of just a very short click" - no this is not what I said. Once the buffer is drained and MPD starts "Decoder is too slow; playing silence to avoid xrun" it never gets back to playing the stream. What I want is it to auto-reconnect in such cases. Even simply stopping would be better than "playing silence" in an endless loop. The stream does not come back by itself. The "very short click" does not happen, its silent and remains so.

"delay of several seconds until it can resume playback" - still better than never resuming playback in my view.

"One single connection attempt can take a minute or so" - in the case I described here it´s way less than a second. Nuber of reconnection tries could be configurable. In my case the first manual stop/start always reconnected immediately.

When shall MPD give up the reconnect? - yes, debateable. Could be configurable for the curl plugin. Current state of affairs is it "gives up" on the first "Decoder is too slow" and never gets back to play.

"relies that your operating system is configured well." Oh well, its a RedHat Enterprise 8 machine, I´m a RHCE and I can assure you there is no exotic network configuration on this machine.

"override those defaults" - well, these defaults do not work for a live stream. That "playing silence" is pointless. At least there should be the option to skip to the next track in such cases. I could then simply add the stream e.g. 3 times for two retries which should be enough for a day of radio listening.

[GK-774]

What exactly is MPD waiting for while it is playing silence? The stream is available as can be seen with a manual reconnect by stop and start. If it wants to continue at the point in the stream where the connection was interrupted this cant work for a live stream obviously, the server won´t deliver data from the past.

[GK-774]

"MPD uses your operating system's default" which default are you referring to?

[GK-774]

Just tested it by deliberately disconnecting the network cable. This triggered "playing silence to avoid xrun" of course at Sep 30 22:14:31. Plugged the cable in again but it´s still ongoing despite the fact that the stream is obviously available: Sep 30 22:25:43 mpd8.hal9000.at mpd[409880]: alsa_output: Decoder is too slow; playing silence to avoid xrun

This simply does not work, I consider that a serious bug when playing live streams.

mpc stop && mpc play and the stream plays almost instantly.

[MaxKellermann]

it never gets back to playing the stream

That's how it looks with your problem. But your logic is the wrong way round. The symptoms you see can be caused by different problems, and implementing the behavior you demand would make lots of MPD users unhappy.

in the case I described here it´s way less than a second

That does not matter for my question. If I implement something, my code applies to all MPD users, not just you. I need to think of all the cases that can happen, not just yours. I need to see the whole picture. The question I asked was a warning about the general case, but you only see your own problem.

You wrote lots of things that ignore the wider picture, you keep referring only to your own specific setup. For example, I don't care if you use RHEL or if you are a RHCE, I don't even know what that is, and I don't care if you have exotic network configuration or not. None of this matters.

What exactly is MPD waiting for while it is playing silence?

For your OS kernel to continue the TCP stream. MPD waits as long as CURL and your kernel's TCP stack are waiting.

The stream is available as can be seen with a manual reconnect by stop and start

Then why does your kernel fail to resume the existing TCP connection? If you're a RedHat expert, you should be able to find out, shouldn't you?

"MPD uses your operating system's default" which default are you referring to?

Whatever your Linux kernel's TCP stack thinks is good for recovering stalled TCP connections. MPD does not interfere with your kernel, it just trusts that your kernel is doing it right.

Just tested it by deliberately disconnecting the network cable.

Then go and analyze the problem that your TCP stack imposes on MPD!

[GK-774]

Dear Max, let me first say there is really no reason to get offensive. That said, "go and analyze the problem that your TCP stack imposes on MPD" is pointing in the wrong direction. Of course the TCP connection will eventually drop because the streaming server will close it after some timeout, this works as designed. If you want your software "play silence" for ever in such cases is your decision.

Now, what appears to be the issue is the low latency playback (default: yes) in snd_usb_audio These apparently are known to produce problems with xruns (https://lore.kernel.org/all/20210929080844.11583-9-tiwai@suse.de/) and the alsa developers eventually decided to have the possibility to switch it off.

Scenarios tested (all with mpd-0.23.10 checked out today):

1) lowlatency on (the default) - network interrupted for a few seconds and plugged in after the occurence of the first xrun entry in the log. Result: MPD plays silence and continues to do so, it does not recover.

2) lowlatency off (options snd_usb_audio lowlatency=0) - network interrupted for a few seconds and plugged in after the occurence of the first xrun entry in the log. Result: MPD plays silence for a short time, recovers and the stream plays again. That´s what we want. (And no problem with the TCP stack obvoiusly).

3) lowlatency off (options snd_usb_audio lowlatency=0) - network interrupted for some time (ca. 30s) Result: MPD plays silence and continues to do so. That is the situation when the connection is gone.

So basically the problem is solved by switching off low latency for snd_usb_audio (it defaults to on). If 3) should not switch to stop instead of flooding the log might be worth a thought but not essential.

Well, thats it. Have a nice day.

[MaxKellermann]

If you feel offended, I better stop discussing your MPD problems with you, because this has no chance going anywhere if your feelings are involved.

[GK-774]

"if your feelings are involved" huh? What kind of nonsense is this? My feelings have absolutely nothing to do with this, it´s your project not mine.

Anyway, once the TCP connection has dropped (by the streaming server, some router, whatever; such drops happen, that´s a fact of life) MPD will eventually start to play silence while waiting for curl to deliver data again. Just the latter will not happen, curls default is not to reconnect and MPD will sit there "playing silence" forever. If you think that´s an elegant solution, that´s fine with me. The documentation says MPD will honor the settings in .curlrc, will see if this makes a working solution possible. Looking at similar reports from the past (all closed without a solution) I understand that listening to live streams is not really the scope of this project, so I have no expectations anyway. Other than that it works perfect (I use it since 0.18.x).