Closed dheijl closed 6 years ago
Thanks for the interesting post. Do you mean 150 milliseconds or 150 microseconds? If it's 150 milliseconds, that is a really long latency and I imagine it would drive the code that looks for missing packets crazy.
The input packets are already handled by a separate reception thread, so you wouldn't be adding any extra functionality, I'm afraid.
So, I have two ideas. The first is for you, if you'd be kind enough, to repeat your experiments but with the log verbosity turned down to 1 and with statistics enabled. The statistics will give an idea of the amount of packet loss, numbers of retries, etc.
The second idea is for me to modify the code to turn off requests for resends of missing packets. Paradoxically, it might improve things. If it does, then we can experiment with different resend-request parameters.
Thanks for the quick response, and yes it's Milliseconds!
~$ ping 192.168.0.135
PING 192.168.0.135 (192.168.0.135) 56(84) bytes of data.
64 bytes from 192.168.0.135: icmp_seq=1 ttl=128 time=54.5 ms
64 bytes from 192.168.0.135: icmp_seq=2 ttl=128 time=77.6 ms
64 bytes from 192.168.0.135: icmp_seq=3 ttl=128 time=4.56 ms
64 bytes from 192.168.0.135: icmp_seq=4 ttl=128 time=123 ms
64 bytes from 192.168.0.135: icmp_seq=5 ttl=128 time=27.4 ms
64 bytes from 192.168.0.135: icmp_seq=6 ttl=128 time=4.52 ms
64 bytes from 192.168.0.135: icmp_seq=7 ttl=128 time=90.2 ms
64 bytes from 192.168.0.135: icmp_seq=8 ttl=128 time=26.3 ms
64 bytes from 192.168.0.135: icmp_seq=9 ttl=128 time=4.84 ms
64 bytes from 192.168.0.135: icmp_seq=10 ttl=128 time=161 ms
64 bytes from 192.168.0.135: icmp_seq=11 ttl=128 time=26.8 ms
64 bytes from 192.168.0.135: icmp_seq=12 ttl=128 time=2.79 ms
64 bytes from 192.168.0.135: icmp_seq=13 ttl=128 time=128 ms
64 bytes from 192.168.0.135: icmp_seq=14 ttl=128 time=22.2 ms
^C
--- 192.168.0.135 ping statistics ---
14 packets transmitted, 14 received, 0% packet loss, time 13013ms
rtt min/avg/max/mdev = 2.791/53.967/161.658/51.382 ms
I know that the receiving already happens in a separate thread, but it does quite some work before queueing the buffer for playing, and I thought that this could be the reason for the missed packets if Airfoil sends bursts of packets. So my idea was to buffer all packets in a seperate thread without any processing at all.
The log with statistics on:
~$ shairport-sync --statistics -v
alsa output device name is "default".
Version: "3.2d23-OpenSSL-Avahi-ALSA-pa-soxr-metadata-sysconfdir:/etc"
statistics_requester status is 1.
daemon status is 0.
deamon pid file is "/var/run/shairport-sync/shairport-sync.pid".
rtsp listening port is 5000.
udp base port is 6001.
udp port range is 100.
player name is "Danny-Amilo-Si-1520".
backend is "(null)".
on-start action is "(null)".
on-stop action is "(null)".
wait-cmd status is 0.
on-start returns output is 0.
mdns backend "(null)".
stuffing option is "0" (0-basic, 1-soxr).
resync time is 0.050000 seconds.
allow a session to be interrupted: 0.
busy timeout time is 120.
drift tolerance is 0.001995 seconds.
password is "(null)".
ignore_volume_control is 0.
volume_max_db is not set
playback_mode is 0 (0-stereo, 1-mono, 1-reverse_stereo, 2-both_left, 3-both_right).
disable_synchronization is 0.
use_mmap_if_available is 1.
output_rate is 44100.
output_format is 3 (0-unknown, 1-S8, 2-U8, 3-S16, 4-S24, 5-S24_3LE, 6-S24_3BE, 7-S32).
audio backend desired buffer length is 0.150000 seconds.
audio backend latency offset is 0.000000 seconds.
audio backend silence lead-in time is -1.000000 seconds. A value -1.0 means use the default.
volume range in dB (zero means use the range specified by the mixer): 0.
zeroconf regtype is "_raop._tcp".
decoders_supported field is 1.
use_apple_decoder is 0.
alsa_use_playback_switch_for_mute is 0.
no special mdns service interface was requested.
configuration file name "/etc/shairport-sync.conf" resolves to "/etc/shairport-sync.conf".
metadata enabled is 0.
metadata pipename is "(null)".
metadata socket address is "(null)" port 0.
metadata socket packet size is "500".
get-coverart is 0.
loudness is 0.
loudness reference level is -20.000000
avahi: avahi_register.
avahi: register_service.
avahi: request to add "_raop._tcp" service without metadata
avahi: service '7C41F9D07FDF@Danny-Amilo-Si-1520' successfully added.
New RTSP connection from 192.168.0.135:55024 to self at 192.168.0.240:5000 on conversation thread 0.
Play connection from user agent "iTunes/7.6.2 (Windows; N;)" on RTSP conversation thread 0.
DACP-ID string seen: "490A9E71034E4782".
Set up play connection from 192.168.0.135 to self at 192.168.0.240 on RTSP conversation thread 0.
No latency has (yet) been specified. Setting 99225 (2.25 seconds) frames as a default.
Output frame bytes is 4.
Output bit depth is 16.
Dithering will be enabled because the output volume is being altered in software
sync error in milliseconds, net correction in ppm, corrections in ppm, total packets, missing packets, late packets, too late packets, resend requests, min DAC queue size, min buffer occupancy, max buffer occupancy
Set initial volume to -18.000000.
New latency: 88200, sync latency: 77175, minimum latency: 0, maximum latency: 0, fixed offset: 11025.
Hammerton Decoder used on encrypted audio.
Output written with RW
-1.0, 210.2, 286.1, 1003, 21, 0, 0, 35, 5660, 186, 232
-1.1, -133.1, 1957.2, 2006, 94, 0, 0, 94, 5483, 181, 232
0.8, -348.4, 1957.2, 3009, 103, 0, 0, 103, 5698, 204, 232
0.7, -521.2, 1404.9, 4012, 122, 0, 0, 122, 5677, 202, 232
0.2, -130.3, 1535.2, 5015, 146, 0, 0, 146, 5910, 202, 230
-0.0, -144.5, 1804.2, 6018, 160, 0, 0, 166, 5520, 201, 231
0.2, 371.0, 1843.9, 7021, 166, 0, 0, 166, 5722, 207, 232
0.0, -283.2, 1693.8, 8024, 166, 0, 0, 166, 5758, 204, 232
-0.5, 175.6, 1161.3, 9027, 172, 0, 0, 172, 5815, 184, 232
-0.7, 150.1, 1107.5, 10030, 175, 0, 0, 213, 5798, 143, 231
2.2, -798.7, 2186.6, 11033, 214, 0, 0, 214, 5550, 204, 232
-1.1, 121.8, 1883.6, 12036, 224, 0, 0, 224, 5693, 204, 231
^Cshutdown requested...
avahi: avahi_unregister.
Request to shut down all rtsp conversation threads
asking playing threads to stop
Playback Stopped. Total playing time 00:01:45.
Thanks for looking into this!
Danny
Switching to an access point with a much weaker signal (and lower throughput) that is not on the powerline gives much better results for latency:
~$ ping 192.168.0.135
PING 192.168.0.135 (192.168.0.135) 56(84) bytes of data.
64 bytes from 192.168.0.135: icmp_seq=1 ttl=128 time=6.63 ms
64 bytes from 192.168.0.135: icmp_seq=2 ttl=128 time=1.70 ms
64 bytes from 192.168.0.135: icmp_seq=3 ttl=128 time=3.87 ms
64 bytes from 192.168.0.135: icmp_seq=4 ttl=128 time=1.73 ms
64 bytes from 192.168.0.135: icmp_seq=5 ttl=128 time=17.0 ms
64 bytes from 192.168.0.135: icmp_seq=6 ttl=128 time=1.67 ms
64 bytes from 192.168.0.135: icmp_seq=68 ttl=128 time=1.45 ms
64 bytes from 192.168.0.135: icmp_seq=69 ttl=128 time=1.43 ms
64 bytes from 192.168.0.135: icmp_seq=70 ttl=128 time=1.49 ms
64 bytes from 192.168.0.135: icmp_seq=71 ttl=128 time=1.84 ms
64 bytes from 192.168.0.135: icmp_seq=72 ttl=128 time=1.86 ms
64 bytes from 192.168.0.135: icmp_seq=73 ttl=128 time=1.59 ms
64 bytes from 192.168.0.135: icmp_seq=74 ttl=128 time=4.17 ms
...snip...
64 bytes from 192.168.0.135: icmp_seq=75 ttl=128 time=1.48 ms
64 bytes from 192.168.0.135: icmp_seq=76 ttl=128 time=1.84 ms
64 bytes from 192.168.0.135: icmp_seq=77 ttl=128 time=1.55 ms
64 bytes from 192.168.0.135: icmp_seq=78 ttl=128 time=1.94 ms
64 bytes from 192.168.0.135: icmp_seq=79 ttl=128 time=1.50 ms
^C
--- 192.168.0.135 ping statistics ---
79 packets transmitted, 79 received, 0% packet loss, time 78129ms
rtt min/avg/max/mdev = 1.093/4.105/91.615/11.002 ms
Statistics:
~$ shairport-sync --statistics
sync error in milliseconds, net correction in ppm, corrections in ppm, total packets, missing packets, late packets, too late packets, resend requests, min DAC queue size, min buffer occupancy, max buffer occupancy
10.6, -1019.5, 1019.5, 1003, 0, 0, 0, 0, 6142, 222, 232
0.8, -560.8, 1268.9, 2006, 0, 0, 0, 0, 5888, 223, 232
-1.8, 17.0, 2617.1, 3009, 0, 0, 0, 0, 5466, 205, 233
-2.6, 385.2, 2367.9, 4012, 0, 0, 0, 0, 5589, 207, 233
1.9, -827.1, 2152.6, 5015, 0, 0, 0, 0, 5669, 223, 232
-1.2, 430.5, 1489.8, 6018, 0, 0, 0, 0, 5743, 223, 232
-0.7, 96.3, 1971.4, 7021, 0, 0, 0, 0, 5633, 224, 233
1.0, -436.2, 2198.0, 8024, 0, 0, 0, 0, 5672, 223, 233
-0.2, -28.3, 2113.0, 9027, 0, 0, 0, 0, 5766, 223, 232
-1.2, 444.7, 1079.1, 10030, 0, 0, 0, 0, 5671, 222, 233
0.6, 209.6, 1960.0, 11033, 0, 0, 0, 0, 5744, 206, 233
2.7, -1288.7, 2195.1, 12036, 0, 0, 0, 0, 5892, 207, 232
-2.0, 776.1, 1489.8, 13039, 0, 0, 0, 0, 5809, 223, 232
0.9, 116.1, 2365.1, 14042, 0, 0, 0, 0, 5802, 223, 233
1.8, -1155.6, 2039.3, 15045, 2, 0, 0, 2, 5537, 222, 232
-1.8, 770.4, 1501.2, 16048, 2, 0, 0, 2, 5532, 223, 233
-0.2, -48.2, 1560.7, 17051, 2, 0, 0, 2, 5627, 223, 233
1.2, -903.5, 2025.2, 18054, 2, 0, 0, 2, 5487, 223, 232
-1.2, 8.5, 1345.4, 19057, 2, 0, 0, 2, 5370, 222, 232
-1.9, 563.6, 1645.6, 20060, 2, 0, 0, 2, 5690, 222, 233
0.4, -161.4, 1311.4, 21063, 2, 0, 0, 2, 5941, 206, 232
0.9, -300.2, 934.7, 22066, 2, 0, 0, 2, 5976, 207, 231
-2.6, -175.6, 1965.7, 23069, 2, 0, 0, 2, 5433, 208, 233
-0.7, 87.8, 1277.4, 24072, 2, 0, 0, 2, 5496, 222, 233
2.2, -167.1, 2030.8, 25075, 2, 0, 0, 2, 5805, 221, 232
-0.7, -410.7, 1849.6, 26078, 2, 0, 0, 2, 5441, 222, 233
-0.6, 682.6, 2195.1, 27081, 2, 0, 0, 2, 5776, 223, 232
0.9, -912.0, 1484.2, 28084, 2, 0, 0, 2, 5652, 222, 232
-0.7, -93.5, 1407.7, 29087, 2, 0, 0, 2, 5516, 222, 232
-2.7, 611.8, 1348.2, 30090, 2, 0, 0, 2, 5568, 223, 233
-2.0, 444.7, 2002.5, 31093, 2, 0, 0, 2, 5383, 222, 234
0.4, -657.1, 946.0, 32096, 2, 0, 0, 2, 5999, 224, 232
0.1, -722.3, 1504.0, 33099, 2, 0, 0, 2, 5577, 222, 232
-2.2, 399.4, 1928.9, 34102, 2, 0, 0, 2, 5512, 204, 233
-0.9, 90.6, 2062.0, 35105, 2, 0, 0, 2, 5585, 206, 233
-1.2, 662.8, 2237.6, 36108, 2, 0, 0, 2, 5743, 224, 233
0.3, -739.3, 1464.4, 37111, 2, 0, 0, 2, 5344, 223, 233
1.4, -339.9, 1133.0, 38114, 2, 0, 0, 2, 6010, 223, 232
^CPlayback Stopped. Total playing time 00:05:11.
So the WiFi latency seems to cause all the problems. I might still have a go at my idea of reading packets without any processing in a separate thread though, and if I succeed (it's been almost 20 years since I did anything in C/C++ and/or Linux/Unix) I will let you know if it changed anything!
Danny
I think my idea wouldn't make any difference, I increased the default Linux UDP buffer size from 16KB to 25 MB and it didn't make any difference, so the missing packets are getting lost elsewhere... I'll use the low latency access point.
Thanks again.
Danny
Thanks. Just incidentally, the third column shows a lot of overcorrection going on. Is this a Linux running in a VM, by any chance?
No, the Linux runs on the physical machine (Linux Lite, core duo 1,8 GHz, 2GB ram). I'm going to increase the drift tolerance as synchronization is not an issue.
Good idea.
It looks like the latest version has solved the Airfoil Windows problems, until recently I still had "static" issues every few seconds (packet loss).
No more sync errors now, and no packet loss. Just the occasional late packet. Could be the resend request on the wrong channel fix.
Thanks for all the good work!
Thanks for the update – that's good news indeed.
Hi,
Thanks for sharing this with us all!
This is with the latest development version of shairport-sync on Ubuntu 16.04 (pulse audio), and the latest Airfoil for Windows, on a WiFi network with excellent throughput but high latency (up to 150 msec, via a powerline WiFi adapter). The "official" version of shairport-sync is totally unusable on this network with Airfoil: every few seconds sound drops out for at least a second, so I gave up. No trouble at all streaming from Itunes from Windows or local Spotify or web-players. When I recently saw an Airfoil mention in the development branch comments history, I decided to give the development version a try. It's much better, the drop-outs only lasting a fraction of a second with an occasional longer lapse and (infrequently) the sound speeding up.
I had already considered forking the github repo to experiment with an asynchronous receive thread for the input stream in player.c in the hope that buffering the Airfoil stream independently from the player logic might fix the missed frames problem (replacing the current network receive logic with a simple get buffer call). Could this potentially fix the latency problem? I am not familiar with the airplay/rtp/rtsp protocols, so messing with that is beyond me I'm afraid.
Synchronization is not an issue for me here, it just has to play the stuff just like Itunes would do it (the Linux laptop is connected to a 2-speaker hifi system with a TOS-link).
The log of a session:
Danny