AirPlay targets show artist, title, and art, but don't seem to know that they're playing music

[airdrummingfool]

Description

When playing music from the built-in Spotify integration to an AirPlay device (using AirPlay 1), the AirPlay device UI (e.g. shown on screen, remote iOS devices, or interaction via Siri) does not indicate that the AirPlay device is playing music. Specifically, the "play/pause" button is a "Play" button, when "Pause" is expected.

This has a few minor negative effects on the AirPlay device UI:

• The AirPlay song progress bar is not consistently updated. It seems to be updated once every ≈25 seconds, which corresponds to a raop: send_progress: event in the logs.
• The "Play/Pause" button UI is incorrect: a "Play" button is shown; since music is playing, I'd expect to see a "Pause" button. If "Play" is tapped, the button turns to "Pause", the progress bar jumps and starts getting updated every second, and music continues. If at that point "Pause" is tapped, the progress bar jumps and goes back to updates every 25 seconds, and the button goes back to "Play", but music continues.
• Asking Siri "What song is this" gets the response "There's nothing playing on this HomePod". If you push "Play" in the AirPlay iOS controls while the music is playing, she will answer subsequent questions about the music correctly.

Reproduction example

Using the built-in Spotify integration in OwnTone I start playing a song and then enable output to my HomePod. The song is played correctly from the HomePod, and the OwnTone UI shows that the song is playing:

If I open the Home app and look at the HomePod details, I see the title, artist, and art for the music, but the app says the HomePod is "Stopped", and the Play/Pause button is a Play button.

Similarly, if I open "Control Other Speakers & TVs" on my iOS device to see what the HomePod is playing, I see the same thing as in the Home app.

If I ask Siri on the HomePod "What song is this", she responds with "There's nothing playing on this HomePod". However, if I return to the Home app on my iOS device and tap the "Play" button and then ask again, she will answer correctly.

Other details

This happens when AirPlaying to a HomePod or to a MBP w/Monterey. I've seen it when playing from the built-in Spotify integration and when playing Spotify Connect through a pipe.

There is a similar bug when using AirPlay 2 (raop_disable = true), but behavior is a bit different:

• Tapping the "Play" button will pause music and show a "Pause" button. Tapping the "Pause" button will start music and show the "Play" button.
• Progress updating is inconsistent in a different way.

Info

• OwnTone version: 28.2
• Platform: Docker container with network_mode: host
• HomePod OS: 15.1.1
• MBP version: Monterey 12.0.1

Thank you for this great project!

[ejurgensen]

Thanks for a very thorough report! I see the same behavior between OwnTone and my ATV4, but interestingly also between iTunes and the ATV4. iTunes still uses Airplay 1. If you have a Windows computer with iTunes you could also try with your Homepod and check.

So if even Apple can't get this right I think it might be a limitation of using Airplay 1 when both speaker and iOS are geared for Airplay 2.

I know OwnTone also issues getting this right with Airplay 2, and here it is more a consequence of me not knowing how it is supposed to work on a technical level.

If you can confirm you have the same issue with iTunes, then I suggest we change this issue to focus on Airplay 2, and then add the "Help wanted" tag. Maybe someone out there can help.

[aleszczynskig]

I have an issue open with Apple on the issue of airplay 1 functionality to airplay 2 devices which doesn't even work with their own products. They have requested more information on the ticket so seem interested to investigate. Hopefully, if and when they fix it themselves, Owntone will just start working too.

[airdrummingfool]

~Sorry for the delay. I was able to test with iTunes (on a Windows box) playing to an ATV (4th gen / HD). I could not reproduce the incorrect play button behavior described above; whenever the music was playing via AirPlay, iTunes correctly showed a pause button. I was able to pause and then continue playback correctly as well.~

Edit: a brain lapse makes this comment irrelevant

[ejurgensen]

iTunes correctly showed a pause button

As I understand this issue, it was not about whether the sender showed correct playback status - it's about what the Home app (and the receiver) shows?

[airdrummingfool]

Whoops, that's what I get for trying to test while on vacation. You're correct - my apologies. I will test again and see what happens in the Home app.

[airdrummingfool]

OK, I re-tested and am seeing similar erroneous behavior from iTunes (on Windows) to a HomePod mini. I agree that it makes sense to shift this issue's focus to AP2.

Edit: is there anything I can do testing-wise to help narrow down the problem?

[ejurgensen]

is there anything I can do testing-wise to help narrow down the problem?

Thanks for the offer, but I can't think of anything. What is required would be a better understand of the protocol.

[aleszczynskig]

Interestingly I was testing today with the latest HomePod OS beta and everything was working as you would expect. I thought that maybe Apple had fixed it. Sadly though it gets in a pickle quite quickly. When I was controlling my HomePod in control centre on my iPhone from an owntone source the controls were working as expected and reflecting the correct state. Once I paused playback via the owntone web ui though, the music stopped but control centre status did not update and was then out of sync with owntone. @ejurgensen I remember back when we were testing the HomePod controls with control centre you triggered owntone to toggle playback on a play or pause event. I wonder if with the latest HomePod OS we should try to respond to the events correctly? Is this worth a try? I know there have been significant changes in HomePod OS (refer to pyatv for some of the protocol details so maybe Apple have fixed this (whether this is intentional or not, I don't know).

I will test with iTunes for Windows now to see if that is working as expected.

Update - no from iTunes in Windows. The controls have no impact and status of iTunes is not reflected in Control Centre.

[ejurgensen]

I remember back when we were testing the HomePod controls with control centre you triggered owntone to toggle playback on a play or pause event. I wonder if with the latest HomePod OS we should try to respond to the events correctly?

OwnTone should already be responding to events with an OK, I added that a while ago. From your observations it could sound like a connection between the controller (the Home app) and OwnTone is needed, so that OwnTone can send events to it. I'm not sure what the procedure for establishing such a connection would be.

refer to pyatv for some of the protocol details so maybe Apple have fixed this

There is a lot of great info in the pyatv repo, but I'm not sure what changes might be especially relevant to know about?

[aleszczynskig]

Sorry I wasn't clear. I didn't mean to acknowledge the events. I mean't to play on a play event and pause on a pause event rather than toggle for either. I'm wondering if this would keep the control centre controls aligned to what owntone is doing. It appears that owntone is ahead of iTunes for Windows now, as you simply cannot control iTunes for windows from control centre.

In regards to pyatv. I was thinking of the companion protocol which is used by HomePods. This is new in HomePod OS 15. I don't think it's the answer, but as pyatv can now control and display metadata for HomePods, I was wondering if there are some parts of the companion protocol that could be woven into owntone to enable better HomePod support from iOS.

[postlund]

The companion protocol is not used to signal any metadata. It is only used for media control in this context, so you can send certain buttons and other requests (play, pause, etc.) but also receive updates regarding availability for some of these commands. It's mostly not that interesting. The companion protocol itself has been around since Handoff and Continuity was introduced, so it's not very new. It's more of a transport protocol, carrying various services. Media control is one of them, which was introduced in iOS 12 or 13 (can't remember which). Before that the remote control widget used MRP directly like the Remote app instead of Companion like it does today.

Regarding metadata and all that... The HomePod runs a stripped version of tvOS from what I have gathered and is very similar to an Apple TV. In tvOS Apple stopped using "native" MRP and only supports tunneling it over AirPlay 2. It uses a special remote control type, but it's still the same old MRP. You will get updates regarding which commands are available or not via MRP, but the main problem still persists as they are reported incorrectly (I.e. pause is for instance reported when something is streaming). I have the exact problem with pyatv and would love a solution, but I don't think you have much to collect down that road I'm afraid.

[aleszczynskig]

@postlund - thanks for the information, my knowledge is very limited, so I'm glad you are able to set the record straight. Currently when playing from owntone to a HomePod, the metadata, including track information and album art is correctly shown in control centre in iOS. The play/pause commands do work but only as a toggle. They normally do not reflect the actual state of playback in owntone. The next and previous track buttons do not seem to work either, I believe they did work on HomePod OS 14, I don't quite remember now (@ejurgensen - did you ever implement these controls from HomePods? I guess if you didn't than my memory is completely wrong). Based upon what you've stated above about the companion protocol, can it be used within owntone to ensure that the playback controls will work as you would expect in control centre, or do the limitations you described mean this is probably going to be broken anyway?

[postlund]

The control center implementation uses Companion, so if it is working as expected in control center then Companion will work. There's one problem though and that is authentication. Companion requires authentication similar to HAP (SRP, Chacha20Poly1305 and all that) which I have reverse engineered. But... I have not found any way to pair with a HomePod, so it's not really possible to use Companion with a HomePod because of that. One guy managed to extract credentials from the key ring on his Mac since they are synchronized, but that's not really a viable option. Did you try if the buttons works as expected via AirPlay/MRP in pyatv (with atvremote)?

[aleszczynskig]

I've just tried with atvremote. Using raop or companion protocols the controls do work however play and pause just toggle owntone, which is the expected behaviour. Regardless, control centre controls on my iPhone do not reflect any of the changes, metadata is correctly displayed. Using airplay the behaviour is different. The first command will pause owntone (presuming it is already playing) subsequent commands start playback on the HomePod from my Apple Music library.

The behaviour of next is bizarre. Using airplay, companion and raop, owntone does select the next track, however playback is paused because the HomePod seems to interpret the command in such a way to switch the HomePod to playing from my Apple Music library. It selects an Apple Music track but does not start playback, this disconnects the HomePod from owntone which subsequently pauses playback to all selected speakers.

A mixed bag, I don't know if there is anything there that holds any promise?

[aleszczynskig]

@ejurgensen - thank you for the new branch. I will report my findings once I've managed to get it compiled and running somewhere.

[ejurgensen]

Thanks a lot for commenting, @postlund, pyatv looks great and you have done an excellent job with the documentation - looks like a goldmine. The companion protocol is completely new to me, a lot to learn there, it seems.

I mean't to play on a play event and pause on a pause event rather than toggle for either.

Gotcha, as you saw I made that change here so you can try it.

The next and previous track buttons do not seem to work either, I believe they did work on HomePod OS 14, I don't quite remember now @ejurgensen - did you ever implement these controls from HomePods?

Yes, I did indeed add that. If it no longer works it must mean that the HomePod's don't send these events any more. If you set the log level to INFO you should see what events are received in the log.

[aleszczynskig]

Sorry for the delay in updating this. I am having some troubles with the new build and I am trying to troubleshoot the issue.

I have an Ubuntu VM running in Parallels on my M1 MacBook. The networking is bridged. Owntone built fine and is playing to one of my HomePods without issue however it is completely unresponsive to atvremote commands or commands direct from the HomePod. I am not seeing anything in the owntone logs even in debug level. I believe that the commands may not be making to the VM but I am unsure why. The current production build on raspberry pi is still working as it was previously. Unfortunately I cannot use my raspberry pi to test this as I don't want to break anything on my "production" system as other members of family depend on it.

Any ideas why the VM might not be receiving these connections from the HomePod?

[ejurgensen]

Any ideas why the VM might not be receiving these connections from the HomePod?

I think it is likely that they are never sent, but as to why that is, I don't know. If you think it is related to the version of OwnTone you could of course try with older versions to see if it makes a difference.

[aleszczynskig]

Yes. I will give it ago

[airdrummingfool]

I just got another HomePod, and instead of updating it right away, I left it at iOS 14.5. Testing shows that my above-described behavior is pretty much the same on 14.5 as 15.1.1.

I did find something potentially interesting - if I try to skip forward using the Control Center, I can get the UI in a state that it actually represents what is happening (play/pause button is "pause" while the music is playing, pause button works to pause and changes to "Play" button, progress bar is ticking upwards and stays within 1s of actual time shown in OwnTone, etc).

This worked on both iOS 14.5 and 15.1.1:

1. Start music playing in OwnTone to a HomePod (in my case I use the built-in Spotify integration)
2. Open Control center, see that play/pause button shows "Play" even though music is already playing
3. Tap >> (track skip forward) in CC. Music playback pauses, but "Play" button remains in CC. (note that OwnTone web UI shows that music is still playing - I believe incorrectly - but the HomePod disconnects in ≈10s)
4. Before 10s is up and the HomePod disconnects, tap "Play" in CC. The next track(!) starts playing, and the "Play" button correctly becomes a "Pause" button. At this point play/pause works and is correct, and the progress bar ticks up every second and is accurate.

If you reverse (3) and (4) above, the music stops and the HomePods disconnect from OwnTone. OwnTone clears the play queue also, almost as if the Spotify component crashes or something.

[aleszczynskig]

Yes this has always been an issue. The underlying issue seems to be that some of the protocols required to make this work correctly are either not understood or not implemented in Owntone.

It's likely this is a broader issue that even impacts use cases for Apple's own products that only implement Airplay 1 I.e iTunes for Windows.

[airdrummingfool]

I should have mentioned, I have set OwnTone to disable raop in order to test AirPlay 2, per earlier discussion that we'd change this issue to focus on AP2.

[aleszczynskig]

Probably the best way to go. At some point I'm sure this will be cracked open. I'm looking forward to that day.

[Kwinnieprince]

I have some same issue, Latest HomePod software (16.2) and latest owntone software (28.5)

When I start playing Spotify (using spoon and PIPE) my HomePod shows the first song correct, with timeline and playing status. After that it stops updating and shows status playing but the songs and timeline do not update anymore.

edit: raop is also disabled