Flac streaming 32bits - ogg_encoder crashes

[FlorianReiterer]

Hello, im trying to stream Flac 96k 24 bit audio to icecast2. If setting 32 bits in my .liq

%ogg(%flac(bits_per_sample=32)

i crash with Source stream failed while streaming: Flac.Internal!

what i already did to come here: i did ogg flac files first (.oga) - but icecast stopped after the first song with a "source timeout" - i guess thats the end-of-file problem. then flac files (.flac). end-of-file solved. then realised i get truncated to 44.1k 16 bit - so i set the flac options in output.icecast - crash with above error.

its weird anyway that 24bits are not available as option. from what i understand 32 bit flac is not even an option for flac?

thanks for looking into it, regards, Florian

heres the part with the error:

2020/01/11 09:43:58 [Musik:3] Prepared "/home/florian/Musik/Mother Earth Radio/Musik//Jose Rodriguez/09 Caminito.flac" (RID 1).
2020/01/11 09:43:58 [random_4965:3] Switch to Musik.
2020/01/11 09:43:58 [clock.wallclock_main:2] Source stream failed while streaming: Flac.Internal!
2020/01/11 09:43:58 [clock.wallclock_main:3] Raised at file "ogg_flac.ml", line 72, characters 24-43
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "ogg_flac.ml", line 76, characters 14-18
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "ogg_formats/ogg_flac_encoder.ml", line 53, characters 18-28
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "ogg_formats/ogg_muxer.ml", line 217, characters 10-41
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "encoder/ogg_encoder.ml", line 226, characters 25-48
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "list.ml", line 77, characters 12-15
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "encoder/ogg_encoder.ml", line 228, characters 6-24
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "outputs/output.ml", line 248, characters 24-44
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "outputs/output.ml", line 259, characters 35-47
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "outputs/output.ml", line 175, characters 6-22
2020/01/11 09:43:58 [clock.wallclock_main:3] Called from file "clock.ml", line 160, characters 17-25

here the liq:

#!/usr/bin/liquidsoap

set("log.stdout", true)

set("frame.audio.channels",2)
set("frame.audio.samplerate",96000)

set("request.metadata_decoders",["FLAC","OGG","TAGLIB"])

Musik = playlist(reload=3600,"~/Musik/Mother Earth Radio/Musik/")

radio = random([Musik])

 output.icecast(%ogg(%flac(samplerate=96000, channels=2, compression=5, bits_per_sample=32)), fallible = true, icy_metadata = "guess",
     host = "localhost", port = 8000,
     password = "XXXXX", mount = "stream",
     name = "Mother Earth Radio", description = "High Resolution Music tuned to 429hz", url = "http://icecast3.streamserver24.com:18800/motherearth", radio)

P.S. just realised i am posting on developer's github. Please excuse if this is no proper report

[autonarcosis]

I can reproduce this error as well using v1.40

2020/01/13 10:48:57 [main:3] Liquidsoap 1.4.0 2020/01/13 10:48:57 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.4.3 sedlex=2.1 menhirLib=20190924 dtools=0.4.1 duppy=0.8.0 cry=0.6.4 mm=0.5.0 ogg=0.5.2 vorbis=0.7.1 mad=0.4.5 flac=0.1.5 flac.ogg=0.1.5 dynlink=[distributed with Ocaml] lame=0.3.3 frei0r=0.1.1 fdkaac=0.3.2 theora=0.3.1 gavl=0.1.6 ffmpeg=0.4.1 alsa=0.2.3 ao=0.2.1 samplerate=0.1.4 taglib=0.3.6 camomile=1.0.2 ladspa=0.1.5 gd=1.0a5

There are two temporary solutions. You could use 16bit in the current configuration or use output.external and ffmpeg to send a 24bit ogg/flac to Icecast. The ffmpeg solution you will not be able to send any sort of metadata.

Example;

output.external(%wav, "ffmpeg -loglevel quiet -re -i pipe:0 -content_type audio/ogg -c:a flac -compression_level 5 -ar 96000 -sample_fmt s32 -f ogg icecast://source:password@URL", radio)

This will produce a ridiculous 1.8 megabit to 3.5 megabit audio stream. My tests with a source 44.1khz sample rate was averaging about 2 megabits.

[FlorianReiterer]

Thank You for the workaround and the reformatting! The data rate is crazy indeed...have to see if my provider is able to cope with that.

Workaround very nice, loss of metadata is a bummer though. Yet i'd think it would be ideal if liquidsoap would just pass through if the audio meets the criteria. like output.icecast(%flac, pass) if previous set=sample_rate etc. matches?

regards, Florian

[FlorianReiterer]

Hi again, i realise i can only use 32bit via ffmpeg (no sample_fmt s24) - and that turns out to be a bit too much to stream it out. Still trying to stream 96k 24bits tough, may i ask for any other ideas?

thanks, Florian

[autonarcosis]

The sample format s32 is 24bit FLAC. FFMPEG explicitly tells you. I highly recommend to start using ffmpeg via the command line and read the documentation to gain more understanding of it. You are not going to avoid the large amount of data this creates. My recommendation is; Don't Bother. Unless the receiving end is going through high end DAC and high end sound systems with the ability to reproduce the dynamic range. To put it in perspective, only about 33 Blu-Ray Disc releases contain 96k/24bit audio tracks which near all of them are live Classical Music performances. There is only one (1) 192k/24bit audio track Blu-Ray Disc release.

[flac @ 0x55ea181ff3c0] encoding as 24 bits-per-sample

Output #0, flac, to 'test.flac': Metadata: encoder : Lavf58.29.100 Stream #0:0: Audio: flac, 96000 Hz, stereo, s32 (24 bit), 128 kb/s Metadata: encoder : Lavc58.54.100 flac

[FlorianReiterer]

thanks for the insight, didnt look close enough (RTFM) while all you stated is true, i still am eager to make this happen. Most of the current Audio DACs in PC's or smartphones etc. seem to be capable of playing 96/24 to the analog Outputs; and i record Vinyls @ 384k/32bits, so the Audio is there, plus, i use the EBU R128 guideline for levelling the radio (thats quite low, about -12dbFS) and therefore would of course benefit from the increased dynamic range.

The ffmpeg ouput config you suggested is at the stream-provider's desk to test; maybe we'll go onair with it soon, i'll let you know the outcome.

regards from munich, Florian motherearthradio.de

[FlorianReiterer]

followup: we are running in 32 bit for a test period of several weeks, then we'll think it all over. you're very welcome to give it a listen, it's a free Service. motherearthradio.de

[toots]

I'll try to have a look at this soon. Quick glance at the code seems to show that it at least was designed to work with sample format up to 32 bits. Internally, we use 64 bits floats so this should work too. Only thing remaining is your input. I'm not sure which decoders support more than the standard S16 sample size. Flac and ffmpeg should, at least, when the underlying files support it.

[autonarcosis]

Toots, perhaps I am looking at old information, but the FLAC format only goes up to 24bits. I know of no encoder or decoder that plays or detects 32bit+ FLAC files. I suspect diminishing returns after 144db of dynamic range. Not many people have horns the size of a house carved from a single block of Rich Corinthian Wood.

The confusion is in how ffmpeg deals with 24bit audio as it only deals with 8, 16, 32... bits. So in order to get a 24bit FLAC you have to tell ffmpeg a sample rate of 32. It's how the encoder muxer works.

[FlorianReiterer]

agreed. FLAC 24bits only - dont know the theory but also my DAW only has 8 16 or 24 bits for FLAC. so i second my own idea - sort of pass through the 24bit FLACs perhaps?

from audio philosphy the improved dynamics of higher bit rates arent that interesting, yet we improve the overall quality of sound wave reproduction by approaching the natural endless-value shaped sound wave. thats how i think of it...and with my low leveling the radio i feel its a good thing to go there. Thanks everyone!

[toots]

According to the FLAC documentation:

FLAC supports from 4 to 32 bits per sample. Currently the reference encoder and decoders only support up to 24 bits per sample. So, 32 bits per sample is possible in the specs but there's no implementation that currently supports it.

If you really want to stream 32 bits, you might have to look at what ffmpeg supports I think, that's your best bet here and, for both encoder and decoder I believe that we receive and send 64bits double floating point samples do you'd be good all the way.

[FlorianReiterer]

Thank you Toots, 24 bits would be actually my intended bitrate - given the even more massive data rate of 32- but the ffmpeg output lacks the metadata, and also crashed on us twice in the last 24hrs, while the FLAC output was perfectly stable and included metadata. So preferably i'd use my 24bit exported FLAC files, and have liquidsoap output via

output.icecast(%ogg(%flac(samplerate=96000, bits_per_sample=32))'

but there's the above error

Source stream failed while streaming: Flac.Internal!

[toots]

Yeah, so for us 32 bits means 32 bits and, accordingly, that fails because it's not implemented in the underlying library.

In fact, we also weren't accepting 24 bits as argument. I just pushed the commit to fix that.

Lastly, you need to set the samplerate globally to 96k, otherwise you're just up-sampling from 44.1kHz:

set("frame.audio.samplerate", 96000)

Also, would you mind sharing your %ffmpeg encoder code? I'd like to make sure that it works as well.

[toots]

Fixed in 3ac68c488639b19fd0b629feefce5e406b3d0609

[FlorianReiterer]

awesome, thank you. Here the ffmpeg .liq

#!/usr/bin/liquidsoap
# Log dir
#set("log.file.path","/tmp/basic-radio.log")
# Log to console
set("log.stdout", true)

# Audio Format
set("frame.audio.channels",2)
set("frame.audio.samplerate",96000)

# Metadaten
set("request.metadata_decoders",["FLAC","OGG","TAGLIB"])

# Pfad zur Musik
Musik = playlist(reload=21600,"/home/florian/Musik/Mother Earth Radio/Musik/")
Klassik = playlist(reload=21600,"/home/florian/Musik/Mother Earth Radio/Klassik/")

radio = random (weights = [1, 9],[Klassik, Musik])

# Ausgang zum Server -FFMPEG 24bit
output.external(%wav, fallible = true, "ffmpeg -loglevel quiet -re -i pipe:0 -content_type audio/ogg -c:a flac -compression_level 5 -ar 96000 -sample_fmt s32 -f ogg icecast://user:pass@icecast3.streamserver24.com:18800/motherearth", radio)

sorry, dont know why it crashed. it ran on the host server, i cant see the logs.

[toots]

Ha ok, external encoder! We've got full support for internal built-in ffmpeg encoder. Here's the syntax in your case:

%ffmpeg(
  format="ogg",
  audio_codec="flac",
  %audio(sample_fmt="s32",compression_level=5)
)

I just tested it, it seems to be working fine!

[FlorianReiterer]

Gents, We're On-Air (or On-Net) with 96k 24bit streaming free music for y'all! Thanks everyone and the savonet devs for making this possible. I m quite proud of you all and this project.

Motherearthradio.de

Love from munich, Florian

[toots]

🎉