karlheyes / icecast-kh

KH branch of icecast
GNU General Public License v2.0
298 stars 107 forks source link

kh13/14 Buffer issues with Muses Player and high bitrates. #303

Open gstephens83 opened 4 years ago

gstephens83 commented 4 years ago

I'm not actually sure why it's doing this. I uninstalled 2.4.4 today due to some connection issues since 2.4.1, and switched to 2.4.0 kh 13. Since then, my 288Kbps 48KHz AAC stream is having buffering issues in Muses web based player that clients use when visiting our web site. I changed the queue-size to 294912 (about 8 seconds) and burst to 147456 (about 4 seconds). These settings previous worked fine in 2.4.4. Without setting them I used to have buffering issues on International listeners to often. These settings work fine playing in VLC, but cause stuttering still in Muses player even when listening via LAN/100Mbps to the server. I started wondering if kh13 was actually obeying those buffer settings. So I installed kh14, same issue. I then removed it, and went back to kh8, problem solved. I see you've introduced seconds for queue and burst sizes. So I tried that before taking kh14 off too. I set 8s queue-size and 4s burst. No luck though, still stutter on some clients yet still not VLC. I tried doubling those figures and still no luck. So I've gone back to kh8 in the meantime like another station here uses. No problems so far.

karlheyes commented 4 years ago

not much to go on. Key things to make sure of, the incoming_bitrate stat is as expected for the stream, there may be some framing issue for the stream I'm not aware of so check for any log messages. I could try relaying the stream if you send me a link to verify the feed. The fact that vlc plays ok would tend to indicate the format and rate is fine but there may be some other aspect that is indirectly tripping things like range requests or headers that the player expects.

karl

gstephens83 commented 4 years ago

I'll try and do some more trouble-shooting with buffer lengths and versions again tonight when I get home from my night job. The direct stream URL is http://icecast.radioinvercargill.nz:8000/radioinvercargill.aac but it's still kh8 running as there's been a few listeners during the day still listening from the web player at http://radioinvercargill.nz . The Muses web player should be running okay though, no hick-ups on kh8 here even with a small buffer over a 4G mobile client. I can change that back to kh13/14 later. It's on the end of a 450Mbps fibre connection. This is the code to launch the player from an html file:

--   |   |   |

karlheyes commented 4 years ago

just tried getting a sample here, granted from the other side of the planet, but after 8-10 secs I'm being dropped. This is with curl against your kh8. I don't see any frame issue in my logging as a relay and the frame spec looks to be something expected.

karl

gstephens83 commented 4 years ago

I think I've found the problem, was steering my in the face after I set up some traffic filters to watch traffic through a router. kh14 is not bursting traffic, kh8 is. This is causing the breakup of traffic during the initial buffer period from the kh14 server. kh8 on the other hand is operating as expected, and bursting very quickly and filling the player up within a split second. So it appears kh14 is ignoring the same burst settings as kh8. I've tried by bytes and time in the format Xs where X is seconds. It's ignoring the burst settings of 4s, kh8 doesn't.

karlheyes commented 4 years ago

don't keep me in suspense, so what setting do you have in the xml?, and what does kh14 error log report at level 4 when you connect the stream?

karl.

gstephens83 commented 4 years ago

Here's the kh14 stream via web player http://radioinvercargill.nz/index2.htm Direct link http://icecast.radioincercargill.nz:7000/radioinvercargill.aac (edit, had wrong port before) Here's the kh8 stream via web player http://radioinvercargill.nz/ Direct link http://icecast.radioincercargill.nz:8000/radioinvercargill.aac

gstephens83 commented 4 years ago

In the XML, apart from host/port/password, the only other changes are to limit to these fields:

<queue-size>589824</queue-size>
<burst-on-connect>1</burst-on-connect>
<burst-size>294912</burst-size>

kh8 handles this, and bursts to client connections as expected. kh14 however does not appear to burst. If it's any less than the above the Muses player starts stuttering and VLC hangs on starting audio (no audio happens but data continues to stream) and never seems to correct for this condition.

The traffic filters on the router I'm using are showing no real traffic above and beyond the normal stream rate on the clients initial connection. kh14 display's slightly less smoothed data rate sent to the client on the LAN. kh8 is very smooth stream rate wise and observes the burst amount correctly at the beginning (a spike of about 2Mbps for a second or less).

So it I'm guessing the issue is the bitrate vs the burst rate in kh13/14. kh8 behaves differently and therefore more suitable to the high-bitrate stream.

Logs to follow

gstephens83 commented 4 years ago

Here is the error log from kh8, no issues at all:

[2020-05-23 23:36:33] INFO thread/ lock abort set to 0 [2020-05-23 23:36:33] INFO main/server_proc_init Icecast 2.4.0-kh8 server reading configuration from icecast.xml [2020-05-23 23:36:33] INFO main/server_process Icecast 2.4.0-kh8 server started [2020-05-23 23:36:33] INFO client/log_commit_thread started [2020-05-23 23:36:33] INFO client/workers_adjust requested worker count 10 [2020-05-23 23:36:33] INFO client/worker_start starting incoming worker thread [2020-05-23 23:36:33] DBUG client/worker 00792dc8 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00792f38 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00785fb0 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 007864e8 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00786380 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00786650 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00786a30 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00786dd0 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00786ee8 now has 0 clients [2020-05-23 23:36:33] DBUG client/worker 00787980 now has 0 clients [2020-05-23 23:36:33] DBUG yp/yp_recheck_config Updating YP configuration [2020-05-23 23:36:33] INFO yp/yp_recheck_config Adding new YP server "http://dir.xiph.org/cgi-bin/yp-cgi" (timeout 15s, default interval 600s) [2020-05-23 23:36:33] INFO yp/yp_client_add Starting Directory client for YP processing [2020-05-23 23:36:33] DBUG client/worker_add_pending_clients Added 1 pending clients to 00787980 [2020-05-23 23:36:33] DBUG client/worker 00787980 now has 1 clients [2020-05-23 23:36:33] INFO connection/get_ssl_certificate No SSL capability on any configured ports [2020-05-23 23:36:33] INFO connection/connection_setup_sockets 2 listener socket(s) for port 8001 [2020-05-23 23:36:33] INFO connection/connection_setup_sockets 2 listener socket(s) for port 8000 [2020-05-23 23:36:33] INFO connection/connection_setup_sockets 4 listening sockets setup complete [2020-05-23 23:36:33] INFO connection/connection_thread connection thread started [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global client_connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global clients (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global listener_connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global source_client_connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global source_total_connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global sources (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global stream_kbytes_read (0) [2020-05-23 23:36:33] DBUG stats/stats_global_calc update global stream_kbytes_sent (0) [2020-05-23 23:36:33] DBUG client/worker 00786f90 now has 0 clients [2020-05-23 23:36:33] DBUG yp/check_servers Add pending yps http://dir.xiph.org/cgi-bin/yp-cgi [2020-05-23 23:36:35] DBUG client/worker_add_pending_clients Added 1 pending clients to 00792dc8 [2020-05-23 23:36:35] DBUG client/worker 00792dc8 now has 1 clients [2020-05-23 23:36:35] DBUG connection/_handle_get_request start with /radioinvercargill.aac [2020-05-23 23:36:35] DBUG fserve/fserve_client_create checking for file /radioinvercargill.aac (.\web\/radioinvercargill.aac) [2020-05-23 23:36:35] WARN fserve/fserve_client_create req for file ".\web\/radioinvercargill.aac" No such file or directory [2020-05-23 23:36:35] DBUG client/client_destroy keepalive detected, placing back onto worker [2020-05-23 23:36:35] DBUG client/worker 00792dc8 now has 0 clients [2020-05-23 23:36:35] DBUG client/worker_add_pending_clients Added 1 pending clients to 00792dc8 [2020-05-23 23:36:35] DBUG client/worker 00792dc8 now has 1 clients [2020-05-23 23:36:35] DBUG stats/stats_global_calc update global client_connections (1) [2020-05-23 23:36:35] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:36:35] DBUG stats/stats_global_calc update global connections (1) [2020-05-23 23:36:36] DBUG stats/stats_global_calc update global outgoing_kbitrate (9) [2020-05-23 23:36:37] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:36:37] DBUG client/worker_add_pending_clients Added 1 pending clients to 00792dc8 [2020-05-23 23:36:37] DBUG client/worker 00792dc8 now has 2 clients [2020-05-23 23:36:37] INFO connection/_handle_source_request Source logging in at mountpoint "/radioinvercargill.aac" [2020-05-23 23:36:37] DBUG source/source_format_init parser found for /radioinvercargill.aac [2020-05-23 23:36:37] INFO format-mp3/format_mp3_get_plugin Created format details for /radioinvercargill.aac [2020-05-23 23:36:37] DBUG stats/stats_handle new source stat /radioinvercargill.aac [2020-05-23 23:36:37] INFO source/source_startup sources count is now 1 [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listenurl" (http://icecast.radioinvercargill.nz:8000/radioinvercargill.aac) [2020-05-23 23:36:37] INFO source/source_apply_mount Applying mount information for "/radioinvercargill.aac" [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listener_peak" (0) [2020-05-23 23:36:37] DBUG format-mp3/format_mp3_apply_settings sending metadata interval 16000 [2020-05-23 23:36:37] DBUG format-mp3/format_mp3_apply_settings charset ISO8859-1 [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "public" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_name" (Radio Invercargill) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_description" (Radio Invercargill, New Zealand. 288 Kbps, 48 KHz, Stereo.) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_url" (http://radioinvercargill.nz) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "genre" (Pop Rock Various Adult Hits Radio) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "bitrate" (288) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_type" (audio/aac) [2020-05-23 23:36:37] DBUG source/source_update_settings max listeners is not specified [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "max_listeners" (unlimited) [2020-05-23 23:36:37] DBUG source/source_update_settings public set to 0 [2020-05-23 23:36:37] DBUG source/source_update_settings queue size to 589824 [2020-05-23 23:36:37] DBUG source/source_update_settings min queue size to 294912 [2020-05-23 23:36:37] DBUG source/source_update_settings burst size to 294912 [2020-05-23 23:36:37] DBUG source/source_update_settings source timeout to 10 [2020-05-23 23:36:37] INFO source/source_startup source /radioinvercargill.aac is ready to start [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "slow_listeners" (0) [2020-05-23 23:36:37] DBUG stats/modify_node_event update "/radioinvercargill.aac" server_type (audio/aac) [2020-05-23 23:36:37] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_peak (0) [2020-05-23 23:36:37] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_peak (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listener_connections" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "stream_start" (23/May/2020:23:36:37 +1200) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "total_mbytes_sent" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "total_bytes_sent" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "total_bytes_read" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "outgoing_kbitrate" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "incoming_bitrate" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "queue_size" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "connected" (0) [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "source_ip" (XXX.XXX.XXX.XXX) [2020-05-23 23:36:37] INFO source/source_init Source /radioinvercargill.aac initialised [2020-05-23 23:36:37] DBUG client/worker_add_pending_clients Added 1 pending clients to 00792dc8 [2020-05-23 23:36:37] DBUG client/worker 00792dc8 now has 3 clients [2020-05-23 23:36:37] INFO source/source_read listener count on /radioinvercargill.aac now 0 [2020-05-23 23:36:37] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listeners" (0) [2020-05-23 23:36:37] DBUG connection/_handle_get_request start with /admin/metadata [2020-05-23 23:36:37] DBUG admin/command_metadata Got metadata update request [2020-05-23 23:36:37] INFO admin/command_metadata Metadata song on /radioinvercargill.aac set to "Radio Invercargill" [2020-05-23 23:36:37] DBUG client/worker 00792dc8 now has 2 clients [2020-05-23 23:36:38] DBUG stats/process_source_stat new node on /radioinvercargill.aac "title" (Radio Invercargill) [2020-05-23 23:36:38] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Radio Invercargill';... [2020-05-23 23:36:38] DBUG stats/process_source_stat new node on /radioinvercargill.aac "yp_currently_playing" (Radio Invercargill) [2020-05-23 23:36:38] DBUG flv/flv_meta_increase 6 array elements [2020-05-23 23:36:38] DBUG stats/process_source_stat new node on /radioinvercargill.aac "metadata_updated" (23/May/2020:23:36:38 +1200) [2020-05-23 23:36:38] DBUG yp/yp_pending_update running through YP changes [2020-05-23 23:36:38] DBUG yp/yp_pending_update Processed 1 changes [2020-05-23 23:36:38] INFO mpeg/check_for_aac detected AAC MPEG-2, rate 48000, channels 2 on XXX.XXX.XXX.XXX [2020-05-23 23:36:38] DBUG stats/process_source_stat new node on /radioinvercargill.aac "audio_codecid" (10) [2020-05-23 23:36:38] DBUG stats/process_source_stat new node on /radioinvercargill.aac "mpeg_samplerate" (48000) [2020-05-23 23:36:38] DBUG stats/process_source_stat new node on /radioinvercargill.aac "mpeg_channels" (2) [2020-05-23 23:36:38] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:36:38] DBUG stats/stats_global_calc update global clients (2) [2020-05-23 23:36:38] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:36:38] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:36:38] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:36:38] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:36:39] DBUG stats/stats_global_calc update global outgoing_kbitrate (1) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (283024) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (70757) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (0) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (70029) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (3) [2020-05-23 23:36:40] DBUG source/source_change_worker moving source from 00792dc8 to 00786f90 [2020-05-23 23:36:40] DBUG client/worker 00792dc8 now has 1 clients [2020-05-23 23:36:40] DBUG client/worker_add_pending_clients Added 1 pending clients to 00786f90 [2020-05-23 23:36:40] DBUG client/worker 00786f90 now has 1 clients [2020-05-23 23:36:40] DBUG client/worker_add_pending_clients Added 1 pending clients to 00792dc8 [2020-05-23 23:36:40] DBUG client/worker 00792dc8 now has 2 clients [2020-05-23 23:36:40] INFO connection/_handle_source_request Source logging in at mountpoint "/x-backup.aac" [2020-05-23 23:36:40] DBUG source/source_format_init parser found for /x-backup.aac [2020-05-23 23:36:40] INFO format-mp3/format_mp3_get_plugin Created format details for /x-backup.aac [2020-05-23 23:36:40] DBUG stats/stats_handle new source stat /x-backup.aac [2020-05-23 23:36:40] INFO source/source_startup sources count is now 2 [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "listenurl" (http://icecast.radioinvercargill.nz:8000/x-backup.aac) [2020-05-23 23:36:40] INFO source/source_apply_mount Applying mount information for "/x-backup.aac" [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "listener_peak" (0) [2020-05-23 23:36:40] DBUG format-mp3/format_mp3_apply_settings sending metadata interval 16000 [2020-05-23 23:36:40] DBUG format-mp3/format_mp3_apply_settings charset ISO8859-1 [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "public" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "server_name" (Radio Invercargill) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "server_description" (Radio Invercargill, New Zealand. 144 Kbps, 48 KHz, Mono.) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "server_url" (http://radioinvercargill.nz) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "genre" (Pop Rock Various Adult Hits Radio) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "bitrate" (144) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "server_type" (audio/aac) [2020-05-23 23:36:40] DBUG source/source_update_settings max listeners is not specified [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "max_listeners" (unlimited) [2020-05-23 23:36:40] DBUG source/source_update_settings public set to 0 [2020-05-23 23:36:40] DBUG source/source_update_settings queue size to 589824 [2020-05-23 23:36:40] DBUG source/source_update_settings min queue size to 294912 [2020-05-23 23:36:40] DBUG source/source_update_settings burst size to 294912 [2020-05-23 23:36:40] DBUG source/source_update_settings source timeout to 10 [2020-05-23 23:36:40] INFO source/source_startup source /x-backup.aac is ready to start [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "slow_listeners" (0) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/x-backup.aac" server_type (audio/aac) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/x-backup.aac" listener_peak (0) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/x-backup.aac" listener_peak (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "listener_connections" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "stream_start" (23/May/2020:23:36:40 +1200) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "total_mbytes_sent" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "total_bytes_sent" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "total_bytes_read" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "outgoing_kbitrate" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "incoming_bitrate" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "queue_size" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "connected" (0) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "source_ip" (XXX.XXX.XXX.XXX) [2020-05-23 23:36:40] INFO source/source_init Source /x-backup.aac initialised [2020-05-23 23:36:40] DBUG client/worker_add_pending_clients Added 1 pending clients to 00792dc8 [2020-05-23 23:36:40] DBUG client/worker 00792dc8 now has 3 clients [2020-05-23 23:36:40] INFO source/source_read listener count on /x-backup.aac now 0 [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "listeners" (0) [2020-05-23 23:36:40] DBUG connection/_handle_get_request start with /admin/metadata [2020-05-23 23:36:40] DBUG admin/command_metadata Got metadata update request [2020-05-23 23:36:40] INFO admin/command_metadata Metadata song on /x-backup.aac set to "Radio Invercargill" [2020-05-23 23:36:40] DBUG client/worker 00792dc8 now has 2 clients [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "title" (Radio Invercargill) [2020-05-23 23:36:40] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Radio Invercargill';... [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "yp_currently_playing" (Radio Invercargill) [2020-05-23 23:36:40] DBUG flv/flv_meta_increase 6 array elements [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "metadata_updated" (23/May/2020:23:36:40 +1200) [2020-05-23 23:36:40] DBUG yp/yp_pending_update running through YP changes [2020-05-23 23:36:40] DBUG yp/yp_pending_update Processed 1 changes [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global client_connections (3) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global clients (3) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global connections (5) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global source_client_connections (2) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global source_total_connections (2) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global sources (2) [2020-05-23 23:36:40] DBUG stats/stats_global_calc update global stream_kbytes_read (68) [2020-05-23 23:36:40] INFO mpeg/check_for_aac detected AAC MPEG-2, rate 48000, channels 1 on XXX.XXX.XXX.XXX [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "audio_codecid" (10) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "mpeg_samplerate" (48000) [2020-05-23 23:36:40] DBUG stats/process_source_stat new node on /x-backup.aac "mpeg_channels" (1) [2020-05-23 23:36:40] DBUG connection/_handle_get_request start with /radioinvercargill.aac [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_connections (1) [2020-05-23 23:36:40] DBUG source/listener_change_worker moving listener 0 from 00792dc8 to 00786f90 [2020-05-23 23:36:40] DBUG client/worker 00792dc8 now has 1 clients [2020-05-23 23:36:40] DBUG client/worker_add_pending_clients Added 1 pending clients to 00786f90 [2020-05-23 23:36:40] DBUG source/locate_start_on_queue XXX.XXX.XXX.XXX Joining queue on /radioinvercargill.aac (99998, 19) [2020-05-23 23:36:40] DBUG client/worker 00786f90 now has 2 clients [2020-05-23 23:36:40] INFO source/source_read listener count on /radioinvercargill.aac now 1 [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" listeners (1) [2020-05-23 23:36:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_peak (1) [2020-05-23 23:36:41] DBUG stats/stats_global_calc update global client_connections (4) [2020-05-23 23:36:41] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:36:41] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:36:41] DBUG stats/stats_global_calc update global outgoing_kbitrate (3) [2020-05-23 23:36:42] DBUG stats/stats_global_calc update global outgoing_kbitrate (684) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" outgoing_kbitrate (0) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" incoming_bitrate (131272) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_read (49228) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_sent (0) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" total_mbytes_sent (0) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" queue_size (48076) [2020-05-23 23:36:43] DBUG stats/modify_node_event update "/x-backup.aac" connected (3) [2020-05-23 23:36:43] DBUG source/source_change_worker moving source from 00792dc8 to 00786ee8 [2020-05-23 23:36:43] DBUG client/worker 00792dc8 now has 0 clients [2020-05-23 23:36:43] DBUG client/worker_add_pending_clients Added 1 pending clients to 00786ee8 [2020-05-23 23:36:43] DBUG client/worker 00786ee8 now has 1 clients [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global outgoing_kbitrate (513) [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global stream_kbytes_read (114) [2020-05-23 23:36:43] DBUG stats/stats_global_calc update global stream_kbytes_sent (0) [2020-05-23 23:36:44] DBUG stats/stats_global_calc update global outgoing_kbitrate (447) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (472) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (282632) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (247309) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (246784) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (246683) [2020-05-23 23:36:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (8) [2020-05-23 23:36:45] DBUG stats/stats_global_calc update global outgoing_kbitrate (408) [2020-05-23 23:36:45] DBUG stats/stats_global_calc update global stream_kbytes_read (286) [2020-05-23 23:36:45] DBUG stats/stats_global_calc update global stream_kbytes_sent (241) [2020-05-23 23:36:46] DBUG stats/stats_global_calc update global outgoing_kbitrate (385) [2020-05-23 23:36:47] DBUG stats/stats_global_calc update global outgoing_kbitrate (369) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" outgoing_kbitrate (0) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" incoming_bitrate (138760) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_read (138766) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_sent (0) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" total_mbytes_sent (0) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" queue_size (137930) [2020-05-23 23:36:48] DBUG stats/modify_node_event update "/x-backup.aac" connected (8) [2020-05-23 23:36:48] DBUG stats/stats_global_calc update global outgoing_kbitrate (358) [2020-05-23 23:36:48] DBUG stats/stats_global_calc update global stream_kbytes_read (374) [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00787980 has 1 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00786f90 has 2 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00786ee8 has 1 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00786dd0 has 0 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00786a30 has 0 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00786650 has 0 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 007864e8 has 0 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00786380 has 0 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00785fb0 has 0 clients [2020-05-23 23:36:48] DBUG client/find_least_busy_handler handler 00792f38 has 0 clients [2020-05-23 23:36:49] DBUG stats/stats_global_calc update global outgoing_kbitrate (348) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (316) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (285192) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (427792) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (427008) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (314903) [2020-05-23 23:36:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (13) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global clients (3) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global connections (5) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global outgoing_kbitrate (342) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global source_client_connections (2) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global source_total_connections (2) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global sources (2) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global stream_kbytes_read (551) [2020-05-23 23:36:50] DBUG stats/stats_global_calc update global stream_kbytes_sent (417) [2020-05-23 23:36:51] DBUG stats/stats_global_calc update global client_connections (4) [2020-05-23 23:36:51] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:36:51] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:36:51] DBUG stats/stats_global_calc update global outgoing_kbitrate (335) [2020-05-23 23:36:52] DBUG stats/stats_global_calc update global outgoing_kbitrate (331) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" outgoing_kbitrate (0) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" incoming_bitrate (141824) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_read (230465) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_sent (0) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" total_mbytes_sent (0) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" queue_size (230084) [2020-05-23 23:36:53] DBUG stats/modify_node_event update "/x-backup.aac" connected (13) [2020-05-23 23:36:53] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:36:53] DBUG stats/stats_global_calc update global outgoing_kbitrate (326) [2020-05-23 23:36:53] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:36:53] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:36:53] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:36:53] DBUG stats/stats_global_calc update global stream_kbytes_read (641) [2020-05-23 23:36:54] DBUG stats/stats_global_calc update global outgoing_kbitrate (322) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (282) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (286248) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (608282) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (607232) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (324086) [2020-05-23 23:36:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (18) [2020-05-23 23:36:55] DBUG stats/stats_global_calc update global outgoing_kbitrate (320) [2020-05-23 23:36:55] DBUG stats/stats_global_calc update global stream_kbytes_read (817) [2020-05-23 23:36:55] DBUG stats/stats_global_calc update global stream_kbytes_sent (593) [2020-05-23 23:36:56] DBUG stats/stats_global_calc update global outgoing_kbitrate (317) [2020-05-23 23:36:57] DBUG stats/stats_global_calc update global outgoing_kbitrate (315) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" outgoing_kbitrate (0) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" incoming_bitrate (142024) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_read (319566) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_sent (0) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" total_mbytes_sent (0) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" queue_size (286852) [2020-05-23 23:36:58] DBUG stats/modify_node_event update "/x-backup.aac" connected (18) [2020-05-23 23:36:58] DBUG stats/stats_global_calc update global outgoing_kbitrate (313) [2020-05-23 23:36:58] DBUG stats/stats_global_calc update global stream_kbytes_read (904) [2020-05-23 23:36:59] DBUG stats/stats_global_calc update global outgoing_kbitrate (311) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (282) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (286304) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (787338) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (786432) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (328747) [2020-05-23 23:37:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (23) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global clients (3) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global connections (5) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global outgoing_kbitrate (310) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global source_client_connections (2) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global source_total_connections (2) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global sources (2) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global stream_kbytes_read (1079) [2020-05-23 23:37:00] DBUG stats/stats_global_calc update global stream_kbytes_sent (768) [2020-05-23 23:37:01] DBUG stats/stats_global_calc update global client_connections (4) [2020-05-23 23:37:01] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:37:01] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:37:01] DBUG stats/stats_global_calc update global outgoing_kbitrate (320) [2020-05-23 23:37:02] DBUG stats/stats_global_calc update global outgoing_kbitrate (281) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" outgoing_kbitrate (0) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" incoming_bitrate (142128) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_read (408638) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_sent (0) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" total_mbytes_sent (0) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" queue_size (294911) [2020-05-23 23:37:03] DBUG stats/modify_node_event update "/x-backup.aac" connected (23) [2020-05-23 23:37:03] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:37:03] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:37:03] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:37:03] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:37:03] DBUG stats/stats_global_calc update global stream_kbytes_read (1165) [2020-05-23 23:37:04] DBUG stats/stats_global_calc update global outgoing_kbitrate (282) [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00787980 has 1 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00786f90 has 2 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00786ee8 has 1 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00786dd0 has 0 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00786a30 has 0 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00786650 has 0 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 007864e8 has 0 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00786380 has 0 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00785fb0 has 0 clients [2020-05-23 23:37:04] DBUG client/find_least_busy_handler handler 00792f38 has 0 clients [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (282) [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (286488) [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (966914) [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (966656) [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (317969) [2020-05-23 23:37:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (28) [2020-05-23 23:37:05] DBUG stats/stats_global_calc update global outgoing_kbitrate (281) [2020-05-23 23:37:05] DBUG stats/stats_global_calc update global stream_kbytes_read (1340) [2020-05-23 23:37:05] DBUG stats/stats_global_calc update global stream_kbytes_sent (944) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" outgoing_kbitrate (0) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" incoming_bitrate (142936) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_read (500292) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" total_bytes_sent (0) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" total_mbytes_sent (0) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" queue_size (294132) [2020-05-23 23:37:08] DBUG stats/modify_node_event update "/x-backup.aac" connected (28) [2020-05-23 23:37:08] DBUG stats/stats_global_calc update global outgoing_kbitrate (282) [2020-05-23 23:37:08] DBUG stats/stats_global_calc update global stream_kbytes_read (1431) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (283) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (287352) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1149416) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (1148928) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (1) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (325176) [2020-05-23 23:37:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (33) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global clients (3) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global connections (5) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global source_client_connections (2) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global source_total_connections (2) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global sources (2) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global stream_kbytes_read (1609) [2020-05-23 23:37:10] DBUG stats/stats_global_calc update global stream_kbytes_sent (1122) [2020-05-23 23:37:11] DBUG stats/stats_global_calc update global client_connections (4) [2020-05-23 23:37:11] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:37:11] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:37:11] DBUG stats/stats_global_calc update global outgoing_kbitrate (281)

Still streaming and burst and buffered fine on kh8.

gstephens83 commented 4 years ago

Log output from kh14, initial buffer problems, lots of stuttering and eventually terminates the client from listening. Same XML configuration but no sign of burst over the Ethernet port.

[2020-05-23 23:49:29] INFO thread/ lock abort set to 0 [2020-05-23 23:49:29] INFO logging/recheck_log_file Using global log file .\logs\access.log [2020-05-23 23:49:29] INFO main/server_proc_init Icecast 2.4.0-kh14 server reading configuration from icecast.xml [2020-05-23 23:49:29] INFO connection/get_ssl_certificate No SSL capability on any configured ports [2020-05-23 23:49:29] INFO connection/connection_setup_sockets 2 listener socket(s) for port 7001 [2020-05-23 23:49:29] INFO connection/connection_setup_sockets 2 listener socket(s) for port 7000 [2020-05-23 23:49:29] INFO connection/connection_setup_sockets 4 listening sockets setup complete [2020-05-23 23:49:29] INFO main/server_process Icecast 2.4.0-kh14 server started [2020-05-23 23:49:29] INFO client/workers_adjust requested worker count 10 [2020-05-23 23:49:29] INFO client/log_commit_thread started [2020-05-23 23:49:29] INFO client/worker_start starting incoming worker thread [2020-05-23 23:49:29] DBUG client/worker 00ae3a30 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00ae3ba0 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00aff558 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00aff5b0 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00af20d8 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00af2180 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00aff608 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00af1fc0 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00af2a10 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00af2a68 now has 0 clients [2020-05-23 23:49:29] DBUG client/worker 00af33f0 now has 0 clients [2020-05-23 23:49:29] DBUG yp/yp_recheck_config Updating YP configuration [2020-05-23 23:49:29] INFO yp/yp_recheck_config Adding new YP server "http://dir.xiph.org/cgi-bin/yp-cgi" (timeout 15s, default interval 600s) [2020-05-23 23:49:29] INFO yp/yp_client_add Starting Directory client for YP processing [2020-05-23 23:49:29] DBUG client/worker_add_pending_clients Added 1 pending clients to 00af33f0 [2020-05-23 23:49:29] DBUG client/worker 00af33f0 now has 1 clients [2020-05-23 23:49:29] DBUG yp/check_servers Add pending yps http://dir.xiph.org/cgi-bin/yp-cgi [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global client_connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global clients (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global listener_connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global source_client_connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global source_total_connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global sources (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global stream_kbytes_read (0) [2020-05-23 23:49:29] DBUG stats/stats_global_calc update global stream_kbytes_sent (0) [2020-05-23 23:49:29] INFO connection/connection_thread connection thread started [2020-05-23 23:49:30] DBUG client/worker_add_pending_clients Added 1 pending clients to 00ae3a30 [2020-05-23 23:49:30] DBUG client/worker 00ae3a30 now has 1 clients [2020-05-23 23:49:30] INFO connection/_handle_source_request Source logging in at mountpoint "/radioinvercargill.aac" [2020-05-23 23:49:30] DBUG source/source_format_init parser found for /radioinvercargill.aac [2020-05-23 23:49:30] INFO format-mp3/format_mp3_get_plugin Created format details for /radioinvercargill.aac [2020-05-23 23:49:30] DBUG stats/stats_handle new source stat /radioinvercargill.aac [2020-05-23 23:49:30] INFO source/source_startup sources count is now 1 [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listenurl" (http://icecast.radioinvercargill.nz:7000/radioinvercargill.aac) [2020-05-23 23:49:30] INFO source/source_apply_mount Applying mount information for "/radioinvercargill.aac" [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listener_peak" (0) [2020-05-23 23:49:30] DBUG format-mp3/format_mp3_apply_settings sending metadata interval 16000 [2020-05-23 23:49:30] DBUG format-mp3/format_mp3_apply_settings charset UTF-8 [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "public" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_name" (Radio Invercargill) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_description" (Radio Invercargill, New Zealand. 288 Kbps, 48 KHz, Stereo.) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_url" (http://radioinvercargill.nz) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "genre" (Pop Rock Various Adult Hits) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "bitrate" (288) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "server_type" (audio/aac) [2020-05-23 23:49:30] DBUG source/source_update_settings max listeners is not specified [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "max_listeners" (unlimited) [2020-05-23 23:49:30] DBUG source/source_update_settings public set to 0 [2020-05-23 23:49:30] DBUG source/source_update_settings source timeout to 10 [2020-05-23 23:49:30] INFO source/source_startup source /radioinvercargill.aac is ready to start [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "slow_listeners" (0) [2020-05-23 23:49:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" server_type (audio/aac) [2020-05-23 23:49:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_peak (0) [2020-05-23 23:49:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_peak (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listener_connections" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "stream_start" (23/May/2020:23:49:30 +1200) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "total_mbytes_sent" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "total_bytes_sent" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "total_bytes_read" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "outgoing_kbitrate" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "incoming_bitrate" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "queue_size" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "connected" (0) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "source_ip" (XXX.XXX.XXX.XXX) [2020-05-23 23:49:30] INFO source/source_init Source /radioinvercargill.aac initialised [2020-05-23 23:49:30] DBUG client/worker_add_pending_clients Added 1 pending clients to 00ae3a30 [2020-05-23 23:49:30] DBUG client/worker 00ae3a30 now has 2 clients [2020-05-23 23:49:30] INFO source/source_read listener count on /radioinvercargill.aac now 0 [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "listeners" (0) [2020-05-23 23:49:30] DBUG connection/_handle_get_request start with /admin/metadata [2020-05-23 23:49:30] DBUG admin/command_metadata Got metadata update request [2020-05-23 23:49:30] INFO admin/command_metadata Metadata song on /radioinvercargill.aac set to "Radio Invercargill" [2020-05-23 23:49:30] DBUG client/worker 00ae3a30 now has 1 clients [2020-05-23 23:49:30] DBUG stats/stats_global_calc update global client_connections (1) [2020-05-23 23:49:30] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:49:30] DBUG stats/stats_global_calc update global connections (2) [2020-05-23 23:49:30] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:49:30] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:49:30] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:49:30] INFO mpeg/check_for_aac Detected AAC MPEG-2, 48000Hz 2 channel(s) 288 kbps on XXX.XXX.XXX.XXX [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "audio_codecid" (10) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "mpeg_samplerate" (48000) [2020-05-23 23:49:30] DBUG stats/process_source_stat new node on /radioinvercargill.aac "mpeg_channels" (2) [2020-05-23 23:49:31] DBUG stats/process_source_stat new node on /radioinvercargill.aac "title" (Radio Invercargill) [2020-05-23 23:49:31] DBUG format-mp3/mp3_set_title icy metadata as StreamTitle='Radio Invercargill';... [2020-05-23 23:49:31] DBUG stats/process_source_stat new node on /radioinvercargill.aac "yp_currently_playing" (Radio Invercargill) [2020-05-23 23:49:31] DBUG flv/flv_meta_increase 7 array elements [2020-05-23 23:49:31] DBUG stats/process_source_stat new node on /radioinvercargill.aac "metadata_updated" (23/May/2020:23:49:31 +1200) [2020-05-23 23:49:31] DBUG yp/yp_pending_update running through YP changes [2020-05-23 23:49:31] DBUG yp/yp_pending_update Processed 1 changes [2020-05-23 23:49:31] DBUG stats/stats_global_calc update global outgoing_kbitrate (5) [2020-05-23 23:49:32] DBUG stats/stats_global_calc update global outgoing_kbitrate (1) [2020-05-23 23:49:33] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (328880) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (123332) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (0) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (122302) [2020-05-23 23:49:34] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (4) [2020-05-23 23:49:34] DBUG source/update_source_stats /radioinvercargill.aac queue size set to 589824 [2020-05-23 23:49:34] DBUG source/update_source_stats /radioinvercargill.aac min queue size set to 294912 [2020-05-23 23:49:34] DBUG source/update_source_stats /radioinvercargill.aac burst size set to 294912 [2020-05-23 23:49:34] DBUG source/source_change_worker moving source /radioinvercargill.aac from 00ae3a30 to 00af2a68 [2020-05-23 23:49:34] DBUG client/worker 00ae3a30 now has 0 clients [2020-05-23 23:49:34] DBUG client/worker_add_pending_clients Added 1 pending clients to 00af2a68 [2020-05-23 23:49:34] DBUG client/worker 00af2a68 now has 1 clients [2020-05-23 23:49:34] DBUG stats/stats_global_calc update global stream_kbytes_read (119) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (310440) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (155223) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (0) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (154573) [2020-05-23 23:49:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (5) [2020-05-23 23:49:35] DBUG stats/stats_global_calc update global stream_kbytes_read (150) [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00af33f0 has 1 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00af2a68 has 1 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00af2a10 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00af2180 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00af20d8 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00af1fc0 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00aff608 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00aff5b0 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00aff558 has 0 clients [2020-05-23 23:49:36] DBUG client/find_least_busy_handler handler 00ae3ba0 has 0 clients [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global listener_connections (0) [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:49:39] DBUG stats/stats_global_calc update global stream_kbytes_sent (0) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (300504) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (338073) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (0) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (305548) [2020-05-23 23:49:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (10) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global client_connections (1) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global connections (2) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:49:40] DBUG stats/stats_global_calc update global stream_kbytes_read (329) [2020-05-23 23:49:43] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (295032) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (516315) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (0) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (308827) [2020-05-23 23:49:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (15) [2020-05-23 23:49:45] DBUG stats/stats_global_calc update global stream_kbytes_read (503) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global listener_connections (0) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:49:49] DBUG stats/stats_global_calc update global stream_kbytes_sent (0) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (292656) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (695062) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (0) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (311023) [2020-05-23 23:49:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (20) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global client_connections (1) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global connections (2) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:49:50] DBUG stats/stats_global_calc update global stream_kbytes_read (678) [2020-05-23 23:49:52] DBUG client/worker_add_pending_clients Added 1 pending clients to 00ae3a30 [2020-05-23 23:49:52] DBUG client/worker 00ae3a30 now has 1 clients [2020-05-23 23:49:52] DBUG stats/stats_global_calc update global clients (2) [2020-05-23 23:49:52] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00af33f0 has 1 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00af2a68 has 1 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00af2a10 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00af2180 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00af20d8 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00af1fc0 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00aff608 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00aff5b0 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00aff558 has 0 clients [2020-05-23 23:49:52] DBUG client/find_least_busy_handler handler 00ae3ba0 has 0 clients [2020-05-23 23:49:53] DBUG connection/_handle_get_request start with /radioinvercargill.aac [2020-05-23 23:49:53] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_connections (1) [2020-05-23 23:49:53] DBUG source/listener_change_worker moving listener 2 on /radioinvercargill.aac from 00ae3a30 to 00af2a68 [2020-05-23 23:49:53] DBUG client/worker 00ae3a30 now has 0 clients [2020-05-23 23:49:53] DBUG client/worker_add_pending_clients Added 1 pending clients to 00af2a68 [2020-05-23 23:49:53] DBUG source/locate_start_on_queue 115.189.98.32 Joining queue on /radioinvercargill.aac (803505, 509388) [2020-05-23 23:49:53] DBUG client/worker 00af2a68 now has 2 clients [2020-05-23 23:49:53] INFO source/source_read listener count on /radioinvercargill.aac now 1 [2020-05-23 23:49:53] DBUG stats/modify_node_event update "/radioinvercargill.aac" listeners (1) [2020-05-23 23:49:53] DBUG stats/modify_node_event update "/radioinvercargill.aac" listener_peak (1) [2020-05-23 23:49:53] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:49:53] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:49:53] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:49:53] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:49:54] DBUG stats/stats_global_calc update global outgoing_kbitrate (265) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (261) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (292160) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (876494) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (67380) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (341768) [2020-05-23 23:49:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (25) [2020-05-23 23:49:56] DBUG stats/stats_global_calc update global outgoing_kbitrate (263) [2020-05-23 23:49:56] DBUG stats/stats_global_calc update global stream_kbytes_read (855) [2020-05-23 23:49:56] DBUG stats/stats_global_calc update global stream_kbytes_sent (65) [2020-05-23 23:49:57] DBUG stats/stats_global_calc update global outgoing_kbitrate (241) [2020-05-23 23:49:58] DBUG stats/stats_global_calc update global outgoing_kbitrate (230) [2020-05-23 23:49:59] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:49:59] DBUG stats/stats_global_calc update global outgoing_kbitrate (236) [2020-05-23 23:49:59] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:49:59] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:49:59] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:50:00] DBUG stats/stats_global_calc update global outgoing_kbitrate (230) [2020-05-23 23:50:00] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:50:00] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:50:00] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (233) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (291568) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1056948) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (210994) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (369414) [2020-05-23 23:50:00] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (30) [2020-05-23 23:50:01] DBUG stats/stats_global_calc update global outgoing_kbitrate (235) [2020-05-23 23:50:01] DBUG stats/stats_global_calc update global stream_kbytes_read (1031) [2020-05-23 23:50:01] DBUG stats/stats_global_calc update global stream_kbytes_sent (205) [2020-05-23 23:50:02] DBUG stats/stats_global_calc update global clients (2) [2020-05-23 23:50:02] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:50:02] DBUG stats/stats_global_calc update global outgoing_kbitrate (221) [2020-05-23 23:50:03] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:50:03] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:50:03] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:50:03] DBUG stats/stats_global_calc update global outgoing_kbitrate (226) [2020-05-23 23:50:04] DBUG stats/stats_global_calc update global outgoing_kbitrate (223) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (214) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (290632) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1235188) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (333877) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (425637) [2020-05-23 23:50:05] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (35) [2020-05-23 23:50:05] DBUG stats/stats_global_calc update global outgoing_kbitrate (217) [2020-05-23 23:50:05] DBUG stats/stats_global_calc update global stream_kbytes_read (1205) [2020-05-23 23:50:05] DBUG stats/stats_global_calc update global stream_kbytes_sent (325) [2020-05-23 23:50:06] DBUG stats/stats_global_calc update global outgoing_kbitrate (216) [2020-05-23 23:50:07] DBUG stats/stats_global_calc update global outgoing_kbitrate (214) [2020-05-23 23:50:08] DBUG stats/stats_global_calc update global outgoing_kbitrate (218) [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00af33f0 has 1 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00af2a68 has 2 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00af2a10 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00af2180 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00af20d8 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00af1fc0 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00aff608 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00aff5b0 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00aff558 has 0 clients [2020-05-23 23:50:08] DBUG client/find_least_busy_handler handler 00ae3ba0 has 0 clients [2020-05-23 23:50:09] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:50:09] DBUG stats/stats_global_calc update global outgoing_kbitrate (212) [2020-05-23 23:50:09] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:50:09] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:50:09] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (210) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (290504) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1416237) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (469032) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (470781) [2020-05-23 23:50:10] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (40) [2020-05-23 23:50:10] DBUG stats/stats_global_calc update global outgoing_kbitrate (215) [2020-05-23 23:50:10] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:50:10] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:50:10] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:50:10] DBUG stats/stats_global_calc update global stream_kbytes_read (1382) [2020-05-23 23:50:10] DBUG stats/stats_global_calc update global stream_kbytes_sent (456) [2020-05-23 23:50:11] DBUG stats/stats_global_calc update global outgoing_kbitrate (214) [2020-05-23 23:50:12] DBUG stats/stats_global_calc update global clients (2) [2020-05-23 23:50:12] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:50:12] DBUG stats/stats_global_calc update global outgoing_kbitrate (213) [2020-05-23 23:50:13] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:50:13] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:50:13] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:50:13] DBUG stats/stats_global_calc update global outgoing_kbitrate (212) [2020-05-23 23:50:14] DBUG stats/stats_global_calc update global outgoing_kbitrate (211) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (210) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (290304) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1596694) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (595752) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (516079) [2020-05-23 23:50:15] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (45) [2020-05-23 23:50:15] DBUG stats/stats_global_calc update global outgoing_kbitrate (208) [2020-05-23 23:50:15] DBUG stats/stats_global_calc update global stream_kbytes_read (1558) [2020-05-23 23:50:15] DBUG stats/stats_global_calc update global stream_kbytes_sent (579) [2020-05-23 23:50:16] DBUG stats/stats_global_calc update global outgoing_kbitrate (212) [2020-05-23 23:50:17] DBUG stats/stats_global_calc update global outgoing_kbitrate (206) [2020-05-23 23:50:18] DBUG stats/stats_global_calc update global outgoing_kbitrate (197) [2020-05-23 23:50:19] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:50:19] DBUG stats/stats_global_calc update global outgoing_kbitrate (200) [2020-05-23 23:50:19] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:50:19] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:50:19] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (199) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (289904) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1775671) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (714039) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (576806) [2020-05-23 23:50:20] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (50) [2020-05-23 23:50:20] DBUG stats/stats_global_calc update global outgoing_kbitrate (198) [2020-05-23 23:50:20] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:50:20] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:50:20] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:50:20] DBUG stats/stats_global_calc update global stream_kbytes_read (1733) [2020-05-23 23:50:20] DBUG stats/stats_global_calc update global stream_kbytes_sent (694) [2020-05-23 23:50:21] DBUG stats/stats_global_calc update global outgoing_kbitrate (202) [2020-05-23 23:50:22] DBUG stats/stats_global_calc update global clients (2) [2020-05-23 23:50:22] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:50:22] DBUG stats/stats_global_calc update global outgoing_kbitrate (197) [2020-05-23 23:50:23] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:50:23] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:50:23] DBUG stats/stats_global_calc update global listeners (1) [2020-05-23 23:50:23] DBUG stats/stats_global_calc update global outgoing_kbitrate (201) [2020-05-23 23:50:24] INFO source/source_queue_advance Client 2 (115.189.98.32) has fallen too far behind (1324218) on /radioinvercargill.aac, removing [2020-05-23 23:50:24] DBUG stats/modify_node_event update "/radioinvercargill.aac" slow_listeners (1) [2020-05-23 23:50:24] DBUG source/source_listener_release Listener 2 leaving /radioinvercargill.aac [2020-05-23 23:50:24] DBUG client/worker 00af2a68 now has 1 clients [2020-05-23 23:50:24] INFO source/source_read listener count on /radioinvercargill.aac now 0 [2020-05-23 23:50:24] DBUG stats/modify_node_event update "/radioinvercargill.aac" listeners (0) [2020-05-23 23:50:24] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:50:24] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:50:24] DBUG stats/stats_global_calc update global outgoing_kbitrate (198) [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00af33f0 has 1 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00af2a68 has 1 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00af2a10 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00af2180 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00af20d8 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00af1fc0 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00aff608 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00aff5b0 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00aff558 has 0 clients [2020-05-23 23:50:24] DBUG client/find_least_busy_handler handler 00ae3ba0 has 0 clients [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (289696) [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (1955463) [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (416319) [2020-05-23 23:50:25] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (55) [2020-05-23 23:50:25] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:50:25] DBUG stats/stats_global_calc update global stream_kbytes_read (1908) [2020-05-23 23:50:25] DBUG stats/stats_global_calc update global stream_kbytes_sent (792) [2020-05-23 23:50:29] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:50:29] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:50:29] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:50:29] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (289496) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (2135059) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (304904) [2020-05-23 23:50:30] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (60) [2020-05-23 23:50:30] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:50:30] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:50:30] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:50:30] DBUG stats/stats_global_calc update global stream_kbytes_read (2084) [2020-05-23 23:50:32] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:50:33] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:50:33] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:50:34] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:50:34] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (293048) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (2316460) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (312581) [2020-05-23 23:50:35] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (65) [2020-05-23 23:50:35] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:50:35] DBUG stats/stats_global_calc update global stream_kbytes_read (2261) [2020-05-23 23:50:35] DBUG stats/stats_global_calc update global stream_kbytes_sent (792) [2020-05-23 23:50:39] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:50:39] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:50:39] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:50:39] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (292496) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (2495254) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (315608) [2020-05-23 23:50:40] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (70) [2020-05-23 23:50:40] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:50:40] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:50:40] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:50:40] DBUG stats/stats_global_calc update global stream_kbytes_read (2436) [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00af33f0 has 1 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00af2a68 has 1 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00af2a10 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00af2180 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00af20d8 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00af1fc0 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00aff608 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00aff5b0 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00aff558 has 0 clients [2020-05-23 23:50:40] DBUG client/find_least_busy_handler handler 00ae3ba0 has 0 clients [2020-05-23 23:50:42] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:50:43] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:50:43] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:50:44] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:50:44] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (292800) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (2675739) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (319473) [2020-05-23 23:50:45] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (75) [2020-05-23 23:50:45] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:50:45] DBUG stats/stats_global_calc update global stream_kbytes_read (2612) [2020-05-23 23:50:45] DBUG stats/stats_global_calc update global stream_kbytes_sent (792) [2020-05-23 23:50:49] DBUG stats/stats_global_calc update global banned_IPs (0) [2020-05-23 23:50:49] DBUG stats/stats_global_calc update global source_relay_connections (0) [2020-05-23 23:50:49] DBUG stats/stats_global_calc update global stats (0) [2020-05-23 23:50:49] DBUG stats/stats_global_calc update global stats_connections (0) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (292928) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (2855452) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (324850) [2020-05-23 23:50:50] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (80) [2020-05-23 23:50:50] DBUG stats/stats_global_calc update global source_client_connections (1) [2020-05-23 23:50:50] DBUG stats/stats_global_calc update global source_total_connections (1) [2020-05-23 23:50:50] DBUG stats/stats_global_calc update global sources (1) [2020-05-23 23:50:50] DBUG stats/stats_global_calc update global stream_kbytes_read (2787) [2020-05-23 23:50:52] DBUG stats/stats_global_calc update global connections (3) [2020-05-23 23:50:53] DBUG stats/stats_global_calc update global client_connections (2) [2020-05-23 23:50:53] DBUG stats/stats_global_calc update global listener_connections (1) [2020-05-23 23:50:54] DBUG stats/stats_global_calc update global clients (1) [2020-05-23 23:50:54] DBUG stats/stats_global_calc update global listeners (0) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" outgoing_kbitrate (0) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" incoming_bitrate (292728) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_read (3035374) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_bytes_sent (815411) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" total_mbytes_sent (0) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" queue_size (304281) [2020-05-23 23:50:55] DBUG stats/modify_node_event update "/radioinvercargill.aac" connected (85) [2020-05-23 23:50:55] DBUG stats/stats_global_calc update global outgoing_kbitrate (0) [2020-05-23 23:50:55] DBUG stats/stats_global_calc update global stream_kbytes_read (2963) [2020-05-23 23:50:55] DBUG stats/stats_global_calc update global stream_kbytes_sent (792)

gstephens83 commented 4 years ago

And here's a couple of snapshots of the traffic on kh8 vs kh14 from task manager, same config. Remote client is the same from an external public IP. Burst is being ignored on kh14 and traffic delivery is a little more hairy. This I think is what is throwing VLC sometimes, and Muses all the time from the kh14 server.

kh8

kh14

Over the lan the burst on kh8 is about 2Mbps, kh14 still none.

karlheyes commented 4 years ago

From the log, I see the burst being set right and in fact the listener being set at the correct point for the burst but there is no sustained rate even for the stream leading to an eventual drop. The pattern looks to follow a limiter kicking in, like max-bandwidth or so-sndbuf options are used or some external limiter like throttling. Can you check the lag values for the listener on the admin page for the stream, and also check the netstat/ss values for the send-Q value (netstat or ss | grep 7000), finally do an strace of icecast with a listener connecting eg strace -tt -o output -ff -p pid of icecast kh14 then connect the listener until it drops, ctrl-C the strace, send me the output.* files (they are text files)

karl

karlheyes commented 4 years ago

Just to confirm, I'm not able to maintain any useful rate here to verify the test, looks like the routing is way too slow to get between us.

karl.

gstephens83 commented 4 years ago

Hi Karl. Sorry I can't be of much more help, I don't have a Linux desktop here or a spare PC to throw it on, they are all Windows, including the Icecast Win32 server. I've setup another mount on kh14 of 64Kbps http://icecast.radioinvercargill.nz:7000/test-64.aac with a queue-size of 131072, burst 65536. I've also created a Muses player page for this http://radioinvercargill.nz:7000/index3.htm. It's able to burst without any major problems at this bitrate. The stream rate still looks quite unstable.

Here's a snapshot of traffic on the 64Kbps KH14 machine:

kh14-64

Earlier I also tested it at 128Kbps with 8s burst and 16s queue this time, and the burst started getting spread out as you say - as if it's throttled. However, this doesn't happen on kh8 at the full 288Kbps.

kh14-128

I've double checked the router, and there is zero throttling on the kh14 Win32 machine. It's has full 100Mbps access to the router. A quick test to fast.com (nearest Netflix cache) is 92Mbps up and 91Mbps down.

karlheyes commented 4 years ago

So it could be windows specific then. The code allows for scaling to many mbit/s so 288k should be easily handled but windows is not a primary platform so there may be some platform specific limiting factor. If you don't have the so-sndbuf setting then that won't be the usual main reason. I presume Windows has TCP window scaling enabled these days. The question is what else could be a factor, at least something to investigate

karl

gstephens83 commented 4 years ago

The burst speed slowing down or throttling starts in the Win32 build at kh9. It doesn't have anything to do with the change log about ". rate averaging update." or ". mpeg parsing updates. Allows for better detection and reporting, identifies mismatches better, and detects a few more frame types. The sync structure is also reduced so should help in caching." ?

karlheyes commented 4 years ago

Can you check whether adding the following setting in kh14 changes things for you

<so-sndbuf>131072</so-sndbuf>

Either in the listen-socket or the mount for the stream

and let me know which windows version you are running on. My iniital suspicion is maybe too large retry timers with limited fixed TCP windows. Using a larger TCP window may reduce those and increase throughput. Seeing that I had trouble with your kh8 then that may confirm that as well. You may also be using auto scaling and your routing is stuffing it and the timing might be too harse for it.

karl.

gstephens83 commented 4 years ago

I had kh14 installed on Win7 Pro 32-bit. I then installed the same 32-bit installer on a Win7 Pro 64 machine where I'm still doing some testing from. Mainly to keep it away from the kh8 host and the public until it behaves.

I put <so-sndbuf>131072</so-sndbuf> under listen socket on kh14. It's bursting fine now. This is on http://radioinvercargill.nz:7000/radioinvercargill.aac @ 288Kbps with an 8 second burst, 16 queue.

I'm listening from VLC over another ISP (3/4G celluar network) back to my fibre connection where the server is located to add some latency and congestion in but it seems okay now. I just checked the network monitor on the kh14 machine and it's bursting five times higher than the stream rate now like kh8.

I ramped up the bitrate to 320Kbps on kh8, and tuned in via 4G to fibre and it seems okay at this end, but I haven't made that listen socket change on it yet.

gstephens83 commented 4 years ago

After two stream start tests on kh14 with that change:

kh14-snd-buf

karlheyes commented 4 years ago

I've just done some tweaking to the retrying timer so that it retries quicker in such cases. See if the behaviour is any better without the so-sndbuf with the attached installer. icecast-2.4.0-kh14.1_win32_setup.exe.gz

In your case, I would suggest having the so-sndbuf more generally even on kh8 to allow for breathing space with the bandwidth on the socket (this is different to your bandwidth on any local network). It saves the application having to excessively retry or check on the socket. It may not need to be 128k (like I posted), maybe 64k, 48k or 32k is plenty but your default seems low and while I don't know the specifics on your win7, windows has always been slow to catch up in this regard. auto-detection on this stuff is not so straightforward. Most unixen have had scaling windows for years.

karl

gstephens83 commented 4 years ago

It's has definitely improved, it's bursting, not to the same peak rate as the tag change. This is without <so-sndbuf>131072</so-sndbuf>:

kh14 1

I'm happy to put KH14 in to service with the tag change now.

karlheyes commented 4 years ago

ok, the lan will burst more because the ping time will be considerably less than th 3/4G. I did not think it was the burst value aspect just the ability to send. You may need to make some finer adjustments depending on your situation but having the so-sndbuf in listen-socket means all incoming clients will have that reserved so 1000s could take a lot of memory. a lesser size will reserve less and only reduce the burst a little. so up to you.

karl.