How to associate cover with meta?

[luckydonald]

Hi Mike,

I have trouble detecting when a new song starts. In my tests the Cover Arts (PICT) were sent before the metadata begin (pbeg). Is there a secure way to associate the image with the metadata?

Because already opening an issue, what do shairports extra codes pfls (doc says play stream flush), daid and acre do?

Thanks!

[mikebrady]

Hi there. Unfortunately, we are at the mercy of the metadata provider, e.g. iTunes, for the timing and sequencing. The contents of the PICT and the metadata enclosed by the mdst and mden tokens is forwarded as soon as it arrives and in the order it arrives. As far as I can tell, there is no unequivocal way of knowing exactly when a new track starts. The metadata and the picture arrive sometime close to the time the new track begins. (BTW, the pbeg token is sent at the beginning of a play session, not at the beginning of a sequence of metadata.)

daid and acre provide the source's DACP-ID and Active-Remote tokens – useful if you want to find the source's remote control and wish to send commands to it. (I need to update the documentation!)

pfls is sent by Shairport Sync whenever the audio stream is paused and flushed and a prsm is sent whenever an audio stream is started or restarted. BTW you may get multiple pfls messages in a row.

[luckydonald]

Thank you for the detailed answer. I guess I misunderstood the pbeg token.

Do you know if the order of coverart and meta is fixed? (I experienced the cover always following the meta block, here with iTunes 12.2.0.145 on a Mac OS 10.9.5., sadly without a 'real' airplay receiver to study) Did you experience something similar? (If I understood correctly you already answered that with 'no', but I want to be sure.) My biggest problem is to find the moment when to delete the old song's meta & cover. I am currently thinking about a timeout, but that can't be the solution, especially when skipping tracks that won't work... Or can I assume that all tags previously filled will be overwritten? (Including empty covers?)

Edit: Also can you elaborate on the Active-Remote token thingy? I understand that shairport can remote control my (iTunes, iPod, iPhone) client? Can shairport do that?

Edit 2: Another thing I noticed, dates in some fields like 'daap.songlastskipdate' (askd) resolves in something 25 Years in the future. (e.g. 2040-02-06 05:28:16). Did you look into these? I assume that iTunes just doesn't provide these and so sends this instead. They are no problem, but maybe you know something about them...

Edit 3: Also, with the fifo named pipe used, when will there be a EOF written? Or was this a mistake on my side, while piping it between computers (the IP is where shairport runs) with

mkfifo /tmp/shairport-sync-metadata;
while true; do ssh 192.168.178.14 "cat /tmp/shairport-sync-metadata" > /tmp/shairport-sync-metadata; echo -e "LOL FAILED, RETRY.\a"; done

Sorry about spamming you with so many questions, but you seem to have better in-depth knowledge then me. And again, thanks for sharing your time and knowledge!

[roblan]

About Active-Remote: http://nto.github.io/AirPlay.html#audio-remotecontrol

short and simple: You can send GET request with Active-Remote header set on http://AIRPLAY_SOURCE:PORT/ctrl-int/1/COMMAND and that's all ;)

The tricky part is getting source address and port - http://www.sugrsugr.com/index.php/airplay-prev-next/ (mDNS) but when You have DACP-ID You can do it ;)

When source is not "simple" AirPlay source (so iTunes, not iPhone) You can use all DACP commands

[mikebrady]

Yes, @roblan puts it well, and let me give you my take on it:

1. Some sources can accept remote control commands – they advertise this by providing a DACP-ID string. The existence of the string indicates that the source can accept commands and the string itself is a unique identifier.
2. To send commands, you need to get the source's IP number and the port number it is listening on. There are various ways of getting the IP number, but the only way to get the port number is to look at mDNS packets that contain the DACP-ID string and the port number.
3. Reading through mDNS packets programatically requires using extra facilities provided by Avahi, but they are not present in the tinysvcmdns package. That would restrict Shairport Sync to using Avahi only, which is IMHO undesirable.
4. Hence, the thought was to provide the DACP-ID as metadata so that people could write a separate app that, e.g. drives a screen, puts artwork and track information up and accepts commands that can be sent back to the source. Maybe @roblan is up to something similar???

Regarding the rest of your questions:

Answer to Edit 2: I'm afraid I don't know too much about the metadata provided by iTunes. It looks like the dates might have been set wrongly on the iTunes computer.

Answer to Edit 3: AFAIR Shairport Sync never closes that pipe, so there shouldn't be an EOF...

[mikebrady]

BTW, thanks for your questions and comments!

[luckydonald]

0 (cover) So you can't think about a better solution either, right?
In the awsome link @roblan got, the meatadata part states:

The RTP-Info header contains a rtptime parameter with the RTP timestamp corresponding to the time from which the metadata is valid.

The examples all have the same rtptime timestamp, for meta and cover. Might that be a way? Or is that a wrong assumption?

1 (active-remote) Thanks, both of you. I'll have a look into these pages.
I am already writing a python library for that metadata pipe, which has an webserver included as example, showing nice and shiny cover art. Now that I know about that handsome feature I should definitely include that remote control in the future. Shairport-decoder (Python)
I am curious: Your choice to make that ip/port lookup external (i.e. by 3rd parties) is the reason there won't be a $ shairport play/pause/last/next command facility?

2 (future tags) This seems to be all special (generated) metadata like last-skipped last-played etc. Still the value is alway set to a value, I assume as explicitly set. (Not undefined, like I would guess with values like 1970). And to my knowledge thats not the maximum possible number either. But I'll keep staring at that values.

3 (EOF) Okey. Might have been ssh having hickups then.

[roblan]

@mikebrady - yup, I'm trying to create a 'website' remote for forked-dappd and shairport-sync (node + html and js), but I don't have time for it lately. I think that's something similar to @luckydonald project but in node, not python :)

[mikebrady]

@roblan that sounds interesting!

@luckydonald in response to your points:

0 (cover) The RTP-Info header contains a rtptime parameter with the RTP timestamp corresponding to the time from which the metadata is valid. The examples all have the same rtptime timestamp, for meta and cover. Might that be a way? Or is that a wrong assumption?

Thanks for bringing this to my attention; it is something I overlooked. There must be some nice way to make use of it.

1 (active-remote) Thanks, both of you. I'll have a look into these pages. I am already writing a python library for that metadata pipe, which has an webserver included as example, showing nice and shiny cover art. Now that I know about that handsome feature I should definitely include that remote control in the future. (Link i paste later on computer; or see my the commit above)

Great -- I look forward to it

I am curious: Your choice to make that ip/port lookup external (i.e. by 3rd parties) is the reason there won't be a $ shairport play/pause/last/next command facility?

Actually it is the other way around -- it's because I don't think it would be suitable for Shairport Sync to have those facilities that the metadata is exported.

2 (future tags) This seems to be all special (generated) metadata like last-skipped last-played etc. Still the value is alway set to a value, I assume as explicitly set. (Not undefined, like I would guess with values like 1970). And to my knowledge thats not the maximum possible number either. But I'll keep staring at that values.

Fair enough

3 (EOF) Okey. Might have been ssh having hickups then.

I just checked -- the pipe is opened when the first item of metadata is to be sent and it is never closed.

[luckydonald]

@roblan I couldn't find the repo of the project you talked about. You are probably using AJAX/json too? Maybe we can bring the API communication to something similar, and then join forces on the Web interface? ... Because I'm not very good at web-ui design ... And a screenshot. too. Just In case someone stumbles about that: Link to the project luckydonald/shairport-decoder. I use a cubietruck (cubieboard 3), and here is how I compiled shairport-sync on it.

It is, well at least, yeah... something, a start, right? (Hope this doesn't go to off-topic, :sweat_smile:)

[mikebrady]

Nice job. I'll look at that timestamp thing over the next few days, promise!

[luckydonald]

I am looking forward to this. Don't hurry yourself. Great work so far already!

[luckydonald]

I don't mean to bug you, but how is the progress?

[mikebrady]

Got a bit snowed-under at work, sorry.

[mikebrady]

So, I had a quick look at this. It should be possible to pick up that timestamp and output it at the start of the metadata sequence – i.e. as an argument to the mdst token.

I could also introduce a new token that is always output just before a PICT, with the timestamp token in it.

Finally, I could send another token with the value of the timestamp of the frame about to be output "now", so you could estimate, within a few milliseconds, when to update the display.

[luckydonald]

I think that doing them as a seperate information token would be a bad aproach. The information should be another attribute of the existing tokens, the way the protocol deliveres it. (If I understood all the above correctly) Having a timestamp first and afterwards the cover or vice versa, you still have to associate to different tokens with each other. Because you already have an xml style I'd prefer to have another <timestamp>123foobar456</timestamp> inside it, or something

[roblan]

@luckydonald - I'm afraid that I will slow You down ;) I'mt rying to use closest JSON to DACP/DAAP format. I will try to publish my work in next couple of days :) promise

[luckydonald]

Yep, JSON is a better choice than XML in my opinion, too. .

[luckydonald]

I'll just bump in here.

[mikebrady]

Hi there. Sorry about the long delay. I have been thinking about how to do this well. Notwithstanding your different view, I've decided to go with the timestamp before and after approach I suggested. I have three reasons: first and foremost, it would potentially break all existing metadata readers if I changed the basic format of code/type/length/data. Second, it's much easier to implement the extra tags. Third, your reader is going to have do some work anyway to associate picture, progress and metadata. So, I've augmented the mdst and mden tags to include the rtptime as data and they now come before and after a PICT item as well as regular metadata. So now, you have metadata sequences, pictures and progress all tagged with the same rtptime tags. Here's an example output from the metadata reader:

"ssnc" "snua": "iTunes/12.3 (Macintosh; OS X 10.11)".
"ssnc" "acre": "1295082003".
"ssnc" "daid": "F18FE780B35CFD8D".
"ssnc" "pbeg": "".
"ssnc" "pfls": "".
"ssnc" "pvol": "-24.78,24.08,0.00,60.00".
"ssnc" "pvol": "-24.78,24.08,0.00,60.00".
"ssnc" "mdst": "1056687241".
Album Name: "Fauré: Requiem And Other Sacred Music".
Artist: "Members Of The City Of London Sinfonia".
Comment: "".
Composer: "Fauré".
Genre: "Classical".
File kind: "Apple Lossless audio file".
Title: "Requiem: Introït Et Kyrie".
Sort as: "".
"ssnc" "mden": "1056687241".
"ssnc" "mdst": "1056687241".
Picture received, length 0 bytes.
"ssnc" "mden": "1056687241".
"ssnc" "prgr": "1056674953/1056687241/1072515673".
"ssnc" "mdst": "1056730537".
Album Name: "Fauré: Requiem And Other Sacred Music".
Artist: "Members Of The City Of London Sinfonia".
Comment: "".
Composer: "Fauré".
Genre: "Classical".
File kind: "Apple Lossless audio file".
Title: "Requiem: Introït Et Kyrie".
Sort as: "".
"ssnc" "mden": "1056730537".
"ssnc" "mdst": "1056730537".
Picture received, length 4766625 bytes.
"ssnc" "mden": "1056730537".
"ssnc" "prgr": "1056656809/1056730537/1072497529".
"ssnc" "prsm": "".
"ssnc" "mdst": "1072526537".
Album Name: "Fauré: Requiem And Other Sacred Music".
Artist: "Members Of The City Of London Sinfonia".
Comment: "".
Composer: "Fauré".
Genre: "Classical".
File kind: "Apple Lossless audio file".
Title: "Requiem: Offertoire".
Sort as: "".
"ssnc" "mden": "1072526537".
"ssnc" "mdst": "1072526537".
Picture received, length 4766625 bytes.
"ssnc" "mden": "1072526537".
"ssnc" "prgr": "1072495817/1072526537/1094792777".

Notice also that the progress information contains the same rtptime. It seems to be sent just at the time a new track is about to be played, so should be good for timing when to update the picture and display.

[luckydonald]

Not quite sure what to do with the reader's output, as I am working with the XML output. So I am confused. Can you give a brief before-after of the changed xml structures?

[mikebrady]

The reader output is very lightly translated from the original XML. Some of the metadata is recognised and translated, for example the Album Name, Artist and Composer tokens. The picture token is also recognised but only the picture length is output. For the tokens it doesn't recognise, it just puts out the type, code and any data there might be.

Let me say what's happening. The important thing to remember is that a numerical quantity called the rtptime can be used to identify the metadata, picture and progress information of a single item, such as a track.

When a sequence of metadata is sent by, say, iTunes, it is tagged with an rtptime. Shairport Sync now sends this rtptime as a piece of data in the mdst item, and also in the mden item which precede and follow, respectively, the metadata items. All the metadata items send between an mdst and an mden should be understood to be "tagged" with this rtptime.

Similarly, when a picture is sent by iTunes, it is tagged with an rtptime. Shairport Sync now sends a pcst and pcen items containing the rtptime before and after the PICT item contain the picture data. The picture should be understood to be "tagged" with this rtptime.

By the way, mdstis mnemonic for MetaData STart, mden for MetaData ENd, pcst for PiCture STart and pcen for PiCtureENd.

The progress item prgr contains three numbers -- the start rtptime, the current item's rtptime and the ending rtptime as before.

In the example above, there are three distinct rtptime tags: 1056687241, 1056730537 and 1072526537. You can see metadata, pictures and progress information associated with each.

I hope this helps.

[roblan]

Hm, I don't see rtptime in mdst, mden, pcst and pcen

I'm using latest shairport-sync (branch development). I've tested it with shairport-sync-metadata-reader. Any ideas?

[luckydonald]

Also, when that works again, can someone post some raw xml output, please?

[roblan]

easiest way (for me) to read raw xml from metadata pipe is

cat /tmp/shairport-sync-metadata

but sure - will do :)

[mikebrady]

Hi @roblan. What source are you sending from, please?

[roblan]

iTunes/12.3.1 (Macintosh; OS X 10.11.1) Offical remote app seems to have problem too with correct cover, so they can be gone.

[mikebrady]

Thanks. The rtptime was missing because I am an idiot -- I used the assignment operator = instead of an equality check operator == in a diagnostic. Such irony! Apologies. I just pushed an update at 2.7.2.

[roblan]

Awesome :) Thanks for quick response!

[mikebrady]

I'm closing this issue and I think it's done. Feel free to reopen if necessary.

[NickSutton]

Hey Mike, did you ever implement LuckyDonalds solution? I have the output in terminal, but can't find any instructions of how to get it into my webpage etc.

I note this thread is quite old now, has there been any further developments on displaying the meta data?

Cheers, Nick

[mikebrady]

Hi Nick. Not sure what you mean – I haven't implemented anything with the metadata except the sample reader. Some others have implemented proper readers, as you'll see from various feeds and comments.

[NickSutton]

Thanks for the reply Mike.

I had a good read of the comments last night and looking at a NodeJS solution following on from some other threads. Steep learning curve!!

Cheers, Nick

[mikebrady]

Great. Keep us posted, please.

[roblan]

@NickSutton - https://github.com/roblan/shairport-sync-reader + socket.io - that's my solution :)

[idubnori]

My shairport-sync-trackinfo-reader contains a web viewer example app which is based on top of .NET Core / WebSocket. Also releases the binaries for Raspberry Pi (linux-arm). Hope this helps too.