Randomly skipping old buffer

[cemkoker]

I have 3LAS running behind a nginx reverse proxy; most of the time it works superbly well... but at some point i start to get a few "kipped buffer because it was too old;" errors and then it all the sudden fills up with this errror and stops playing.

Reloading the stream manually works; but i was wondering if we could 'somehow' restart the stream automatically after a number of 'too old' buffers; i've tried adding " this.NextScheduleTime = 0.0;" after the Logger.Log but has no effect.

Any proper way of doing it ? Thanks !

[JoJoBond]

This can have multiple causes e.g.:

• Samples are being discarded that should actually be played
• Network can't keep up, packages arrive late
• Decoder can't keep up, samples arrive late

What format and sample rate are you using?

To reset scheduling you should do it like this: this.NextScheduleTime = this.Audio.currentTime + this.StartOffset;

[cemkoker]

It happens both on wav and mpeg / 16000 I will try to implement your suggestion when it occurs; for me it's ok if there's s small gap as long as I can resume.

[JoJoBond]

At 16kHz there is going to be a sample rate conversion on the client side. Samples may be discarded in that process. I'd need to know more about the client device and browser to figure out a way to recreate this though. Feel free to try out if resetting the scheduling will be fine for your application.

[cemkoker]

Sorry for the late reply. The client can be both desktop/mobile; currently tested with Chrome / Safari it works quite well. You said it's going to cause sample rate conversion; what do you mean ? I was just using the sample example provided. What sample rate would you recon not use conversion ?

After testing, the reset scheduling works quite well but I wonder if there's even a better way.

Thank you !

[JoJoBond]

Your client device is very likely not running its audio system on a 16kHz sample rate (Wikipedia). So when you call decodeAudioData on the AudioContext it will do two things: Decode the compressed data into raw sample data and afterwards convert these samples into new samples based on the sample rate that the AudioContext runs on. The sample rate of the AudioContext cannot be changed, it is defined by the OS/browser. It can be read from the AudioContext.sampleRate property (see https://developer.mozilla.org/de/docs/Web/API/AudioContext). Common sample rates are 44.1kHz and 48kHz, depending on device and OS. Multiples of these values will usually work fine as well (e.g. 96kHz or 192kHz). Going from 16kHz to 48kHz is trivial (note the 1:3 ratio) as you could simply do a linier interpolation to generate the two missing samples (fast and easy, but low quality). However going from 16Khz to 44.1kHz is not so trivial and requires some more complicated steps (not going into detail here). What's important, is that you may end up with a buffer that has a shorter or longer playback time compared the the original encoded audio data. Over a longer period of time this may cause initial buffer to run empty and then the stuttering starts. Resetting the scheduling just resets the initial buffer but it will keep running empty as the actual problem persists.

[cemkoker]

I understand now; what's best then ? Have multiple samplerate streams and use one depending on the client's platform ?

[JoJoBond]

Well, that's possible but kind of overkill. Can you give me the full ffmpeg/avconv command line that you use to feed the server? This way I might be able to reproduce the issue and maybe find a solution.

[cemkoker]

The one from your example (except not using hardware but a multicast input):

ffmpeg -y -f alsa -i hw:0 -rtbufsize 64 -probesize 64 \
-acodec pcm_s16le -ar 16000 -ac 1 \
-f wav -fflags +nobuffer -packetsize 384 -flush_packets 1 - \
| node stdinstreamer.js -port 9602 -type wav -chunksize 384

[JoJoBond]

How would that multicast input scenario look like?

[cemkoker]

I have a Digigram streaming my audio to the multicast; here's the input that i transcode to wav

Duration: N/A, start: 0.000000, bitrate: 64 kb/s
Stream #0:0: Audio: mp3, 48000 Hz, mono, s16p, 64 kb/s

So that's my full launch code:

ffmpeg -y -i rtp://@$IP:5004 -rtbufsize 64 -probesize 64 \
-acodec pcm_s16le -ar 16000 -ac 1 \
-f wav -fflags +nobuffer -packetsize 384 -flush_packets 1 - \
| node stdinstreamer.js -port $PORT -type wav -chunksize 384

[JoJoBond]

Thanks, I'll see if I can find something. This might take some time though.

[JoJoBond]

I can't reproduce the issue with the sine wave test.

I suspect that your RTP stream might have package losses. Can you stream with a protocol that uses TCP instead of UDP?

[cemkoker]

Hi. Sorry for the late reply. No unfortunately I can't change to TCP; but I doubt I have package loss there. What's weird is that it happens randomly, it's hard to reproduce. Is there another way of reseting the stream rather than "this.NextScheduleTime = this.Audio.currentTime + this.StartOffset;" ? Reloading the whole page works fine but not very conviniant.

[JoJoBond]

It doesn't have to be package loss, UDP guarantees neither package delivery nor correct order of packages on the receiving side. On a mp3 stream you wouldn't necessarily notice package loss, as the packages have no connection to each other (if bit reservoir is disabled, which it should be). A 'full' reset needs two things. Call Reset on the LiveAudioPlayer instance and call PurgeData on the AudioFormatReader instance. The "this.NextScheduleTime = this.Audio.currentTime + this.StartOffset;" solution is on for within the PushBuffer method of the LiveAudioPlayer. FFmpeg should show a speed of 1x all the time (it might start a bit lower but should converge to 1x), if it's lower all the time then you probably had package loss.

[cemkoker]

Trying out mp3 as per your suggestion; my source feed's samplerate is 48kHz and bitrate 64kb/s (MPEGL3); i transcode using : /usr/bin/ffmpeg -y -i rtp://@$IP:5004 -rtbufsize 64 -probesize 64 \ -acodec libmp3lame -b:a 48k -ac 1 -reservoir 0 \ -f mp3 -write_xing 0 -id3v2_version 0 -fflags +nobuffer -flush_packets 1 - \ | /usr/bin/node stdinstreamer.js -port $2 -type mpeg I hear a crackling noise (that's not present using Wav).

As for the player; i have buttons to trigger change of the source; each time i do call the Init function in main.js but this creates a new stream; and I end up having 2 streams (or more if I click again).

I would like to be able to stop the stream (close the socket) and start again when Init'ing again. There's no such 'websocket' close function to close it or maybe have a singleton pattern for it ?

[JoJoBond]

For now I don't have any way to disconnect. You'd have to call the disconnect method on the WebSocket member within the WebSocketClient.

But to come back to the issues with the wav. I want to eliminate the possibility that your input stream might have package loss. So the best way to test for stability is to use local source. Just for the purpose of testing. If it's an issue on the client side, it should still show up. If it doesn't show up, that would indicate an issue with your streaming input. I tested it with a signal generator input with no issues on Firefox and Chrome. I also have it in productive use fed from an audio input, again with no issues as far as I can tell.

[cemkoker]

Thanks, solved the disconnection problem. Will investigate for the source; any idea about the crackling/poping audio ? It's because of the different bitrates ?

[JoJoBond]

I'm not entirely sure where that noise comes from, can you try if increasing settings["mpeg"]["MinDecodeFrames"] ? For Android it is set to 17, for everything else it will be set to 3. Check the static method DefaultSettings() of AudioFormatReader, it's set there.

[cemkoker]

Switching to mp3 solved the problem definitely. Thank you !