OGG/Opus metadata disappears when live streaming using input.harbor()

[technicaldesign]

Hi,

I'm currently testing LS 2.0.0+git@62dd6679 on armhf Debian 11 (`thanks a million for the build, however I suspect this problem might be carried on from previous versions EDIT: NOT TRUE). I'm having an issue with (non-ICY) metadata when kicking in a live stream using the input.harbor() method model, which doesn't occur if using the (less appropriate) input.http() method. The live stream is encoded in OGG/Opus, which has native in-stream support for metadata. Parsing all the issues, I've bumped into a very old one from 2016:

https://github.com/savonet/liquidsoap/issues/335

where something similar is reported.

I underline the fact that if I stream live to a separate Icecast mount and then set up my final stream.liq like this:

full = fallback(track_sensitive=false, [input.http("http://localhost:8000/live"), radio])

when I provide such stream, the metadata is updated correctly as reflected both in my web player and VLC.

However, when I use the recommended configuration for live streams:

full = fallback(track_sensitive=false, [input.harbor("live",port=8002,user="xxxx",password="yyyy"), radio])

and stream with my client (which is IDJC Internet DJ Console) to this LS endpoint, the audio is picked up and sent to output/Icecast server correctly, but metadata disappears entirely. If I disconnect and revert to the playlist, metadata works again.

My stream is encoded in OGG/Opus CBR 128Kb, with native metadata (not using the ICY workaround for formats that don't support metadata natively) and metadata encoding is UTF-8.

BTW this is my output line:

output.icecast(%ogg(%opus), host="www.mydomain.eu",port=8000,user="xxxx",password="yyyy", mount="stream",full)

I'm trying to interpret the comments you make to certain possibly related issues:

https://github.com/savonet/liquidsoap/issues/1729#issuecomment-877859769

https://github.com/savonet/liquidsoap/issues/1435#issuecomment-751723623

but I can't come to a conclusion. It seems like you are acknowledging the difference between ICY and native metadata and giving for granted that the latter should be in the stream, but then you also state that FFMPEG doesn't support in-stream metadata encoding so, perhaps, it can't process the metadata correctly (but why, then, when LS uses the input.http() method to fetch the stream, everything is fine? In both cases LS is re-encoding a new stream - otherwise certain transitions wouldn't be possible - so what is the difference? I'm really baffled by this different behavior.). As a newbie, I'm very confused on the issue and, BTW, after parsing the docs, I've also tried to add the following lines to the .liq but with no success:

settings.tag.encodings.set(["UTF-8", "ISO-8859-1"]) settings.protocol.ffmpeg.metadata.set(true) settings.encoder.metadata.export.set(["artist", "title", "album"])

and, finally, I tried:

icy_metadata="true"

in the output (as you have suggested), but that doesn't work (which should be expected, since my player is waiting for native and not for ICY metadata) and in this case metadata disappears even from audio playing from the playlist.

In the APIs, I didn't find any parameter that is explicitly related to the issues, except for icy_metadata="true/false" and, specifically, related to input.harbor().

I'm lost and this is a real showstopper for my application. Can you please tell me what I'm doing wrong or if the input.harbor() method actually prevents OGG/Opus metadata "passthrough". I would be very grateful for any workaround and apologies if the answer is self-evident, but I really tried my best on docs and issues.

Thank you for your awesome work and help,

Stefano

[technicaldesign]

Actually my assumptions were entirely wrong. input.harbor() works fine with metadata in 1.4.x, however the behavior appears to be different in 2.0 (now RC1) where Opus native metadata from an outside source just vanishes using input.harbor() (not input.http()). Investigating further, I have found this discussion on the mailing list that could be related:

https://www.mail-archive.com/savonet-users@lists.sourceforge.net/msg14676.html

Could the issue have something to do with the conversion between the LS (new/abstract?) frame format and FFMPEG's frame format? Also, a pass_metadata parameter that has to be set to true is also mentioned in the discussion but it is unclear where to do so, if other than for filters. I'll try a couple of experiments and, if unsuccessful, revert back to 1.4.x.

[toots]

Hi,

I haven't had much time to check this report but it's.possible the FFmpeg stream decoder in 2.0.0 does not properly return ogg metadata. I'd suggest to force the stream decoder to use the OGG decoder. You should be able to do that using the settings for decoder priorities.

[technicaldesign]

Thank you for your kind reply and suggestion. It seems I can't really set the priority for the OGG decoder. What happens is the following in 2.0-rc1 on armhf. I set (according to the dev docs, but, perhaps I'm doing something wrong):

settings.decoder.priorities.ogg.set(1)

but when the input.harbor() stream kicks in, still no metadata is passed. When I terminate the stream, however, the log displays:

2021/09/04 10:31:00 [input.harbor_0:2] Feeding stopped: Ffmpeg_decoder.End_of_file.

so it seems like the OGG decoder was never used in the first place. I also played with the:

settings.request.metadata_decoders.set(["OGG"])

and

settings.decoder.file_extensions.ogg.set(["ogv", "oga", "ogx", "ogg", "opus"])

and the result was that the metadata wasn't displayed even when playing the playlist files (which still works in 2.0-rc1. I understand that this is telling the FFmpeg decoder to refrain processing metadata and delegating a different decoder (which appears to be non-existent) for the specific task.

finally, I also reverted to default (no decoder setting) and tried to set:

settings.protocol.ffmpeg.metadata.set(true)

but this didn't help either.

I'm now trying to figure out why the OGG decoder is not being used. BTW, in the dev docs (in 1.4.3 I found this: set("decoder.file_decoders",["META","WAV","AIFF","MIDI","IMAGE","FFMPEG","FLAC","AAC","MP4","OGG","MAD","GSTREAMER"]))there doesn't seem to be a way in 2.0 to force the usage of a certain decoder, but only to set a priority, so perhaps it's falling back to FFMPEG for some reason or it's ignoring the command.

Just for your info, on LS startup, I get this information from which it appears to me the OGG libraries are present and recognized, perhaps it's useful:

2021/09/04 10:50:47 >>> LOG START 2021/09/04 10:50:41 [main:3] Liquidsoap 2.0.0-rc1 2021/09/04 10:50:41 [main:3] Using: bytes=[distributed with OCaml 4.02 or above] pcre=7.4.6 sedlex=2.4 menhirLib=20210419 curl=0.9.1 dtools=0.4.4 duppy=0.9.2 cry=0.6.5 mm=0.7.2 xmlplaylist=0.1.5 lastfm=0.3.3 ogg=0.7.0 ogg.decoder=0.7.0 vorbis=0.8.0 vorbis.decoder=0.8.0 opus=0.2.0 opus.decoder=0.2.0 speex=0.4.0 speex.decoder=0.4.0 mad=0.5.1 flac=0.3.0 flac.ogg=0.3.0 flac.decoder=0.3.0 dynlink=[distributed with Ocaml] lame=0.3.4 shine=0.2.2 frei0r=0.1.2 fdkaac=0.3.2 theora=0.4.0 theora.decoder=0.4.0 ffmpeg-avutil=1.0.1 ffmpeg-avcodec=1.0.1 ffmpeg-avdevice=1.0.1 ffmpeg-av=1.0.1 ffmpeg-avfilter=1.0.1 ffmpeg-swresample=1.0.1 ffmpeg-swscale=1.0.1 bjack=0.1.6 alsa=0.3.0 ao=0.2.3 samplerate=0.1.6 taglib=0.3.7 ssl=0.5.9 magic=0.7.3 camomile=1.0.2 inotify=2.3 yojson=1.7.0 faad=0.5.0 soundtouch=0.1.9 portaudio=0.2.2 pulseaudio=0.1.4 ladspa=0.2.0 dssi=0.1.3 tsdl=v0.9.8 tsdl_ttf=0 tsdl_image=0 camlimages=4.2.6 cohttp-lwt-unix=4.0.0 prometheus-app=1.1 srt.constants=0.2.0 srt.types=0.2.0 srt.stubs=0.2.0 srt.stubs.locked=0.2.0 srt=0.2.0 lo=0.2.0 gd=1.0a5

I just wanted to stress again that when connecting a live source using input.http(), the metadata is passed. Isn't FFmpeg decoding used in that scenario too (asking as a complete newbie)?

Also, I wanted to point out (I haven't before), that my playlist files are all MP3s with metadata and the same exact MP3s are used as source for the IDJC client which then re-encodes the OGG/Opus (preserving the metadata, which, again, I stress is passed on when using the input.http() method in 2.0-rc1 and using both input.http() and input.harbor() in the stock Debian 11 1.4.3 package.

I will try to figure out what is going on, but it seems I'm on a dead end.

Thanks for everything, all the best.

Stefano

[toots]

Hi,

Streams are decoded using mime types so the settings are:

settings.decoder.mime_types.ffmpeg.set([...])

(thanks for pointing this one out, we had a duplicate of it in the standard library that was hiding the internal one, it's fixed now!)

So, you can either set this value to an empty list:

settings.decoder.mime_types.ffmpeg.set([])

(but make sure you update to the latest main or remove this line: https://github.com/savonet/liquidsoap/commit/7af6b1c1fb8c1531982117a30e1d3d0ca01b6eaf)

Or, else, force the priority of the ogg decoder to be above ffmpeg's (which is 10):

settings.decoder.priorities.ogg.set(15)

[technicaldesign]

You nailed it! Issue solved by using:

settings.decoder.priorities.ogg.set(15)

My fault for getting the priority value the other way around (thought 1 was some sort of boolean value to override default FFmpeg). Now the OGG decoder is triggered successfully and metadata is passed correctly using input.harbor() method. On live stream disconnection the log shows proper OGG decoder usage:

2021/09/04 12:12:33 [input.harbor_0:2] Feeding stopped: Ogg.End_of_stream.

Awesome, I would still try to point out this variation in behavior of the FFmpeg decoder in the docs or perhaps in the release notes, since most new users will be leaving FFmpeg as default from 1.4.x to 2.0, if this won't be aligned in 2.0 final. Also, I'm afraid I can't change the issue title to reflect it's a 2.0 pre-final issue only. Will close this issue since you're now aware of it and will know how to deal with it best.

Thanks a million for your help!

All the best,

Stefano

[toots]

Glad to hear! We still have a bug tho!

[technicaldesign]

Apologies for closing, got carried away by the success. Being tagged as "usage" I didn't consider this issue to be designated as a bug and your suggestion not really a workaround but a solution, given that I also misunderstood your words as the glitch referring to (updated?) FFmpeg libraries which weren't strictly part of Liquidsoap. The other issue you pointed outEDIT: which mentioned this one, I had parsed it among others but mistakenly considered it unrelated to mine which was strictly OGG specific and there was also that input.http() vs. input.harbor() mismatch that left me scratching my head.

Thanks! Stefano