karlheyes / icecast-kh

KH branch of icecast
GNU General Public License v2.0
297 stars 107 forks source link

Inline metadata not working #368

Closed doug-hoffman closed 1 year ago

doug-hoffman commented 2 years ago

Sending inline metadata from a client appears to be broken beginning with 2.4.0-kh11. A few instances are processed, but then it goes completely wrong:

[2022-05-02  20:44:21] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:21] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:22] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:23] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';...
[2022-05-02  20:44:44] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';ning...';...
[2022-05-02  20:44:51] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';UUStreamTitle='Scanning...';...
[2022-05-02  20:45:37] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';e='Scanning...';...
???X??05-02  20:48:18] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';!k??(?4
      O"@%StreamTitle='TO:200...
[2022-05-02  20:48:20] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';UUUUUStreamTitle='Scanning...';...
[2022-05-02  20:48:27] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='TO:11 Transport FROM:2307';UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU...

Packet captures confirm that valid metadata is being sent at the time it breaks. The client is SDRTrunk, which works fine with icecast-2.4.4 and 2.3.2kh32.

If streaming is allowed to continue after metadata processing breaks, the daemon eventually becomes unresponsive.

rustyhodge commented 2 years ago

Have you done a raw packet capture to see what the triggering metadata is? Look for "StreamTitle" in the raw captured data.

You can use this script to do that (change the URL of course). Tweak max filesize to your preferences, it will depend on the stream bitrate.

#!/bin/bash

source="http://ice.somafm.com/illstreet-128-mp3"
maxseconds=60

target="streamrip-$$.mp3"
echo "statrting rip or $source to $target"
curl -s --user-agent MozillaPro --header 'Icy-MetaData:1' -m $maxseconds -i $source > $target

Run hd on that file, scroll through and you'll see something like this:

00000060  af 12 15 70 6f 07 53 74  72 65 61 6d 54 69 74 6c  |...po.StreamTitl|
00000070  65 3d 27 48 75 67 6f 20  4d 6f 6e 74 65 6e 65 67  |e='Hugo Monteneg|
00000080  72 6f 20 2d 20 54 68 65  6d 65 20 46 6f 72 20 54  |ro - Theme For T|
00000090  68 72 65 65 27 3b 53 74  72 65 61 6d 55 72 6c 3d  |hree';StreamUrl=|
000000a0  27 68 74 74 70 3a 2f 2f  73 6f 6d 61 66 6d 2e 63  |'http://somafm.c|
000000b0  6f 6d 2f 6c 6f 67 6f 73  2f 35 31 32 2f 69 6c 6c  |om/logos/512/ill|
000000c0  73 74 72 65 65 74 35 31  32 2e 6a 70 67 27 3b 00  |street512.jpg';.|
000000d0  00 00 00 00 00 00 c7 57  a5 b1 7d 63 7f 1b d6 a3  |.......W..}c....|

That way you can see if something strange is being sent to the Icecast server that's causing it trouble.

Here's a good resource on the ICY Metadata protocol: https://web.archive.org/web/20070117080413/http://www.smackfu.com/stuff/programming/shoutcast.html

doug-hoffman commented 2 years ago

I have looked at a packet capture. No issues in the actual metadata being sent. Works fine with icecast official and older versions of the kh branch.

I'm not aware of many clients sending inline metadata from the source to server, so it's probably a less tested feature, but should work.

rustyhodge commented 2 years ago

Perhaps I misunderstood, when you say "from the source to server" do you mean the metadata isn't embedded in the audio stream but rather being updated via a link like icecastserver.com:/admin/metadata?mount=/stream&mode=updinfo&song=Scanner+P41+Channel+2 and that's what is causing the issue? (from https://icecast.org/docs/icecast-trunk/admin_interface/#metadata-update )

I'm testing updates done that way every 3 seconds right now, moving between several random words and a shorter phrase. I'm testing the display in Apple's Music.app on a mac, and it's been working for 25 or so iterations as I type this.

Can you give an example of the text you're trying to set?

I'm testing with kh15.

doug-hoffman commented 2 years ago

It's being sent inline, not with separate http requests.

Sample metadata: TO:11 Transport FROM:2307

rustyhodge commented 2 years ago

What is it sending for ServerURL and Stream Title, and how does your encoder set that?

So far I can't duplicated this, using both inline metadata and the URL method above.

Something in your encoder seems to be triggering this.

We are running lots of mounts (the repeats you see below are for multiple bitrate mounts with the same metadata updates), and here's what my logs look like with debug level set:

[2022-09-19  18:20:39] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 1';StreamUrl='http://somafm.com/logos/512/xmasro...
[2022-09-19  18:20:39] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 1';StreamUrl='http://somafm.com/logos/512/xmasro...
[2022-09-19  18:20:40] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='The Enchanters - Let There Be Love';StreamUrl='http://somafm.com/lo...
[2022-09-19  18:20:40] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='The Enchanters - Let There Be Love';StreamUrl='http://somafm.com/lo...
[2022-09-19  18:20:41] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 1';StreamUrl='http://somafm.com/logos/512/xmasro...
[2022-09-19  18:20:41] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Iron & Wine - Belated Promise Ring';StreamUrl='http://somafm.com/lo...
[2022-09-19  18:20:42] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 2';StreamUrl='http://somafm.com/logos/512/xmasin...
[2022-09-19  18:20:42] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='The Enchanters - Let There Be Love';StreamUrl='http://somafm.com/lo...
[2022-09-19  18:20:42] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 2';StreamUrl='http://somafm.com/logos/512/xmasin...
[2022-09-19  18:20:42] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 1';StreamUrl='http://somafm.com/logos/512/xmasro...
[2022-09-19  18:20:43] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 2';StreamUrl='http://somafm.com/logos/512/xmasin...
[2022-09-19  18:20:44] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Heights of Abraham - E.V.A. (Instrumental)';StreamUrl='http://somaf...
[2022-09-19  18:20:44] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Terence Boylan - Hey Papa';StreamUrl='http://somafm.com/logos/512/s...
[2022-09-19  18:20:45] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Terence Boylan - Hey Papa';StreamUrl='http://somafm.com/logos/512/s...
[2022-09-19  18:20:45] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Terence Boylan - Hey Papa';StreamUrl='http://somafm.com/logos/512/s...
[2022-09-19  18:20:45] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Terence Boylan - Hey Papa';StreamUrl='http://somafm.com/logos/512/s...
[2022-09-19  18:20:45] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Heights of Abraham - E.V.A. (Instrumental)';StreamUrl='http://somaf...
[2022-09-19  18:20:46] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Heights of Abraham - E.V.A. (Instrumental)';StreamUrl='http://somaf...
[2022-09-19  18:20:49] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Heights of Abraham - E.V.A. (Instrumental)';StreamUrl='http://somaf...
[2022-09-19  18:20:49] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Somafm - Nov Beg 10';StreamUrl='http://somafm.com/logos/512/covers5...
[2022-09-19  18:20:49] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Somafm - Nov Beg 10';StreamUrl='http://somafm.com/logos/512/covers5...
[2022-09-19  18:20:49] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Terence Boylan - Hey Papa';StreamUrl='http://somafm.com/logos/512/s...
[2022-09-19  18:20:50] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Somafm - Nov Beg 10';StreamUrl='http://somafm.com/logos/512/covers5...
[2022-09-19  18:20:50] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM - Xmas Over 2';StreamUrl='http://somafm.com/logos/512/xmasin...
[2022-09-19  18:20:58] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Somafm - Nov Beg 10';StreamUrl='http://somafm.com/logos/512/covers5...
[2022-09-19  18:20:59] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/xmasro...
[2022-09-19  18:20:59] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/xmasro...
[2022-09-19  18:21:00] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/christ...
[2022-09-19  18:21:00] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/christ...
[2022-09-19  18:21:01] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='SomaFM Fluid - Short ID9';StreamUrl='http://somafm.com/logos/512/fl...
[2022-09-19  18:21:01] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/christ...
[2022-09-19  18:21:01] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/christ...
[2022-09-19  18:21:01] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Haaj - Jingle Jingle';StreamUrl='http://somafm.com/logos/512/christ...
doug-hoffman commented 2 years ago

Here's the relevant bit of code that builds the inline metadata:

    public static String formatInline(String title)
     {
         title = "StreamTitle='" + title + "';";
         int chunks = (int)Math.ceil((title.length() + 1) / 16.0d);
         int nulls = (chunks * 16) - title.length();
         char[] padding = new char[nulls];

         StringBuilder sb = new StringBuilder();
         sb.append((char)chunks).append(title).append(padding);
         return sb.toString();
     }

Bitrate is 8kbps and metadata interval of 1000 bytes, so it's sending once/second since radio transmissions tend to be very short and require frequent changes.

rustyhodge commented 2 years ago

Is that Icecast code or encoder code?

Out of curiosity have you tested to see if this happen with longer metadata intervals?

I think this might be related to the code that kh added in src/mpeg.c which I think was being put in place to support HLS streaming (which requires segments to be broken on codec frame boundaries). I'm in over my head on this, but I think that's where to look and it would be easy to test if increasing the interval fixes it, and at what point that interval size is.

I may experiment with that myself later and report back. Sadly KH seems to have vanished.

karlheyes commented 1 year ago

I have not been able to reproduce this case. There may be something unrelated causing memory corruption which may of been fixed or it may be contingent on some combo of hardware and software setup exposing and issue. At the moment, I do not see what could expose an issue with the metadata interval. Can you verify again with the setup to see if the issue continues with the latest master build (basically kh16.1 plus a log update)

doug-hoffman commented 1 year ago

This issue actually appears to be the result of extraneous data in the non-metadata portion of the stream that causes icecast to look for the metadata in the wrong position.

In DSheirer/sdrtrunk#1244, I reworked MP3 streaming to look for and parse MP3 frame headers, and then dump data into the stream one frame at a time, which (amongst other things) fixed an issue with non-MP3 data being sent in the MP3 portion of the stream. Prior to this, each audio segment was recorded to an individual MP3 file, which was then dumped into the stream in its entirety. I never dug into what the extraneous data was, but I'm guessing ID3 tags.

Without DSheirer/sdrtrunk#1244, I am still able to reproduce the problem within minutes. Once I apply that PR, I cannot.

karlheyes commented 1 year ago

I'll be releasing a kh17 soon enough. If extra data or less data is provided between the icy blocks then obviously playback and relay alignment will be stuffed. Main problem is the ability to resync on the icy block is near impossible. While you could match on say StreamTitle to work it out, that may not appear for some considerable time as most times it's just 1 byte of 0.