philippe44 / AirConnect

Use AirPlay to stream to UPnP/Sonos & Chromecast devices
Other
3.44k stars 217 forks source link

removing unresponsive player #400

Closed systemcrash closed 1 year ago

systemcrash commented 2 years ago

Hello, and thanks for a nice product.

I've been working a lot with AP1/2 in another repo, so I have a good understanding of many of the problems you've faced, except for this one. Is there a way to prevent problems like this on the local side?: removing unresponsive player

I'm trying to get as low latency as possible, and I am in control of the LAN segments and everything is cabled so lower latency is achievable, although I am stuck around ~50 msec (<latency>55::5</latency>). I wanted to get even lower, but even with this, it feels like the latency is longer than the actual latency specified. Around 40msec and the removals seem to start. Is the removal active (us) or passive (sonos device)? Re-starting the binary and sometimes the renderer does not appear (despite it in the auto-generated config xml), does this depend on some full-stack round-trip time for a minimum delay before it's accepted?

Would implementing e.g. a CoreAudio device improve the situation?

What's a theoretical minimum?

The only settings I've changed are wav and latency as above, for the binary ./airupnp-osx-multi

philippe44 commented 2 years ago

Removal and latency should not be related, unless I've left a bug that creates an unwanted connection. Still, there is no point making latency super low

systemcrash commented 2 years ago

What counts as "super low"? If there was no point to low latency, users such as myself would not be trying it :wink:

philippe44 commented 2 years ago

I mean by "no point" that the latency cannot be predictable. And making the RTP latency too low exposes you to lots of hiccups.

systemcrash commented 2 years ago

Jitter is a known problem. But the only jitter is on local-host, if the local host connects to the device. Otherwise, cabled connections are quite good. Latency becomes unpredictable when we're talking milliseconds.

Back to my original question (since you have some familiarity with the stack) - can you suggest a theoretical minimum?

philippe44 commented 2 years ago

I was not talking about jitter but, not knowing precisely what you want to achieve, to the fact that you can't expect any kind of reliable "synchronization" using AirConnect because I do RTP-to-HTTP and the HTTP part (UPnP/CC) has no way to control rendering time at the DAC.

So regarding the RTP minimum time, I don't recommend going below 500ms. What happens is that after [RTP] delay, I start to "feed" the HTTP side. If the player starts at the first received byte (like Sonos do), counting on the fact that because it's HTTP, the resource should be available and there will be a large "burst" of data to come, then there is only disappointment to expect because the only buffer you can expect is [RTP] delay.

Of course, if you have an end-to-end application like VoIP where less than 100ms delay is required, it's different because both sides know when to drop packets/frames. Here, the HTTP side thinks that everything received is fine and just plays it, so you end up, with no buffer, in that silence vs starving dilemma and that builds up, unfortunately

By default, AirPlay1 assumes a 1s (sometimes 2s) RTP buffer

systemcrash commented 2 years ago

All airplay versions (i.e. the specs) assume a 2 second realtime buffer. The apparent reason for this is backwards compatibility with the airport express: the device has limited memory, that in total, with all packets in memory, could hold up to slightly more than a 2 second sliding window of compressed audio packets. At what point in the buffer you decide to start playback (and how much of the incoming buffer you skip) is up to the implementation, but anything other than the 'standard' is noticeable in deviation from other devices playing simultaneously.

Okay, I appreciate the explanation. I've not seen any problems when I send to localhost and configure a 50msec latency buffer, and upnp to my Sonos system. Lower still seems achievable. With even this low latency, I have no loss or jitter. This assumes that all of the airplay happens locally, on the same system, not over any net link.

One advantage of AP2 here is buffered streaming - everything goes via TCP, so you can skip a ton of packet-resend logic, and the sender will prime the receivers buffer.

I guess the one way is to ask Sonos to implement AES67. They already did PTP, so AES isn't that much more of a stretch.