search

FDH2 / UxPlay

AirPlay Unix mirroring server
GNU General Public License v3.0
1.53k stars 78 forks source link

Sound hickups (was due to usb network device problem) #265

Closed arampak closed 8 months ago

arampak commented 8 months ago

I have a problem with the sound crackling once in a while. When run with Gstreamer in debug mode by setting the export GST_DEBUG=2 , it warns of a sync issue with the sound card, and the resync is performed. I am running UxPlay on the recent version of Xubuntu, compiled from sources.

At the same time, there is no such issue with the Strawberry music player that uses the same Gstreamer engine. I am streaming the music from iTunes on iPad and iPhone. I have tried with different sinks, that does not seem to influence anything. I have also used both built-in sound card and the external USB card, the outcome is the same.

fduncanh commented 8 months ago

For listening to itunes, the option "uxplay -async" may be better. This only affects ALAC (audio-only) streaming.

let us know if this fixes your issue.

You can add it to the startup file, and switch it off with the command line option "uxplay -async no" The async option uses timestamps and enforces an itunes latency of 2.0sec. Without it, audio is played when it arrives.

arampak commented 8 months ago

For listening to iTunes, the option "uxplay -async" may be better. This only affects ALAC (audio-only) streaming.

Thank you for your prompt reply! This is fixing the issue to some extent, minimizing the hiccups, but not eliminating them totally. They would still happen sometimes at the very start of the song or when I am doing something on the computer. Still, I can live with that. BTW am I right that whatever the original sound quality is in iTunes, it is being sent to UxPlay as 44K/16 bit stream?

The problem is much more severe when the screen of the iPad is mirrored, and a YouTube video is being played, making that mode totally useless. Is that only me? The network connection is very good, I am using a 5GHz wifi, and the video itself is very smooth, with no issues at all.

One other thing that I noticed is that when I select alsasink, I can't specify the device to play on. UxPlay seems to use the default one. When I run Gstreamer directly, I am able to specify the device (and again, the same is possible explicitly in Strawberry, where I chose the particular hardware device and thus I am in control of what is being sent to that device, avoiding downmixing the sound to 44000/16 by ALSA's pcm virtual device).

fduncanh commented 8 months ago

sound with mirrored video is fine usually, so if not, its only you. audio in mirror mode is AAC audio in audio-only mode is ALAC Airplay uses 44100/16/2

https://gstreamer.freedesktop.org/documentation/alsa/alsasink.html?gi-language=c

try "uxplay -as "alsasink device=...." etc to see if you can specify the sound device

take a look in the Windows section of the README where something similar is done with wasapisink

arampak commented 8 months ago

This is really ugly. I am running everything on a vanilla installation of xubuntu, and the problem persists on both the pretty standard internal Intel sound card (ALC294) as well as the USB device, which is m2tech hiFace Two usb to spdif adapter that presents itself as a standard USB analog sound card. If I run a Gstreamer pipeline directly with a sound file as an input, everything works perfectly well. I would consider the wifi being unstable for some reason, however, in the same setup Shairport works perfectly well without any issues, so the problem seems to be with Gstreamer somehow. The debug warnings from Gstreamer about sync issues with the sound card point at the same direction. I have used the "device=" string already with alsasink but the problem remained, so I thought it didn't work. I am totally lost as to what to try next.

fduncanh commented 8 months ago

(1) you report that in audio-only ALAC mode there are "glitches" in sound and GST_DEBUG=2 reports soundcard sync errors. so post your GST_DEBUG output.

you report option -async helped but not completely.

maybe try pulsesink?

(2) you report that in mirror mode, the AAC audio on youtube videos is so bad that it is "useless"??? any GST_DEBUG for this? uxplay is well tested on ubuntu, and works great (for most users), so you need to debug your system somehow.

arampak commented 8 months ago

Thank you for your reply. Here are the messages I get in pure ALAC mode after I start the session with the iPad. First, I get these as I connect:

$ export GST_DEBUG=2 && uxplay
UxPlay: reading configuration from  /home/aram/.config/uxplayrc
UxPlay 1.68: An Open-Source AirPlay mirroring and audio-streaming server.
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
using system MAC address <hidden>
any AirPlay audio cover-art will be written to file  /home/aram/.cache/uxplay/cover.jpg
Initialized server socket(s)
Accepted IPv4 client on socket 30
Local: 192.168.83.41
Remote: 192.168.83.180
connection request from iPad (iPad12,1) with deviceID = <hidden>

Client identified as User-Agent: AirPlay/750.14.1
Accepted IPv4 client on socket 32
Local: 192.168.83.41
Remote: 192.168.83.180
ct=2 spf=352 usingScreen=0 isMedia=1  audioFormat=0x40000
start audio connection, format ALAC 44100/16/2
raop_rtp starting audio
0:00:38.389468786  3698 0x7fe0880638c0 WARN                    alsa conf.c:5668:snd_config_expand: alsalib error: Unknown parameters {AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:00:38.389554908  3698 0x7fe0880638c0 WARN                    alsa pcm.c:2664:snd_pcm_open_noupdate: alsalib error: Unknown PCM default:{AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:00:38.394722224  3698 0x7fe0880638c0 WARN         audio-resampler audio-resampler.c:274:convert_taps_gint16_c: can't find exact taps

Then, as the hiccups happen, below are the typical messages. There is almost always one in the very beginning of each track, and sometimes they also happen in the middle. I don't seem to be able to figure out any system here. Here is the one with rather many issues:

==============Audio Metadata=============

Album: Hands On
Artist: Jeff Hamilton Trio
Genre: Jazz
Title: 3.000 Miles Ago

coverart size 130551 written to /home/aram/.cache/uxplay/cover.jpg
0:09:32.007035913  3698 0x7fe0880638c0 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink1> Unexpected discontinuity in audio timestamps of +0:00:00.526780045, resyncing
0:09:37.839617557  3698 0x7fe0880638c0 WARN           audiobasesink gstaudiobasesink.c:1491:gst_audio_base_sink_skew_slaving:<alsasink1> correct clock skew -0:00:00.020000307 < -+0:00:00.020000000
==============Audio Metadata=============

Album: Hands On
Artist: Jeff Hamilton Trio
Genre: Jazz
Title: 3.000 Miles Ago

0:10:31.717046941  3698 0x7fe0880638c0 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink1> Unexpected discontinuity in audio timestamps of +0:00:00.271405895, resyncing

Now, when I turn into the screen mirroring mode, below are the messages. As the screen sharing starts, here is what I get right away, without even running any video:

Accepted IPv4 client on socket 30
Local: 192.168.83.41
Remote: 192.168.83.180
connection request from iPad (iPad12,1) with deviceID = <hidden>

Client identified as User-Agent: AirPlay/750.14.1
Accepted IPv4 client on socket 32
Local: 192.168.83.41
Remote: 192.168.83.180
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline
0:12:38.262636788  3698 0x55f7ef4fd6a0 ERROR               glwindow gstglwindow_gbm_egl.c:234:draw_cb: Could not set DRM CRTC: Permission denied (13)
0:12:38.276464784  3698 0x55f7ef4fd6a0 ERROR               glwindow gstglwindow_gbm_egl.c:297:draw_cb: Could not initialize GBM surface
0:12:38.296584627  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:37.745116304 deadline:0:12:37.745116304 earliest_time:0:12:38.265892352
0:12:38.306981219  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:37.778438637 deadline:0:12:37.778438637 earliest_time:0:12:38.265892352
0:12:38.318352941  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:37.945050304 deadline:0:12:37.945050304 earliest_time:0:12:38.265892352
0:12:38.324076058  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:37.978372637 deadline:0:12:37.978372637 earliest_time:0:12:38.265892352
0:12:38.329405568  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.011695012 deadline:0:12:38.011695012 earliest_time:0:12:38.265892352
0:12:38.333806147  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.045017304 deadline:0:12:38.045017304 earliest_time:0:12:38.265892352
0:12:38.337878258  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.078339637 deadline:0:12:38.078339637 earliest_time:0:12:38.265892352
0:12:38.341079698  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.144984304 deadline:0:12:38.144984304 earliest_time:0:12:38.265892352
0:12:38.474700291  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.178317637 deadline:0:12:38.178317637 earliest_time:0:12:38.265892352
0:12:38.483471428  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.211650971 deadline:0:12:38.211650971 earliest_time:0:12:38.265892352
0:12:38.491703656  3698 0x55f7ef4fc980 WARN            videodecoder gstvideodecoder.c:3670:gst_video_decoder_clip_and_push_buf:<avdec_h264-0> Dropping frame due to QoS. start:0:12:38.244984304 deadline:0:12:38.244984304 earliest_time:0:12:38.265892352

Then, as I run a regular Youtube video, here is what happens:


ct=8 spf=480 usingScreen=1 isMedia=1  audioFormat=0x1000000
start audio connection, format AAC-ELD 44100/2
raop_rtp starting audio
0:17:23.363324347  3698 0x7fe08806d800 WARN                    alsa conf.c:5668:snd_config_expand: alsalib error: Unknown parameters {AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:17:23.363376958  3698 0x7fe08806d800 WARN                    alsa pcm.c:2664:snd_pcm_open_noupdate: alsalib error: Unknown PCM default:{AES0 0x02 AES1 0x82 AES2 0x00 AES3 0x02}
0:17:24.191793725  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.032630385, resyncing
0:17:24.656276628  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.391814058, resyncing
0:17:31.255362834  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.000136054, resyncing
0:17:32.273624875  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.000090702, resyncing
0:17:32.364130128  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.010861678, resyncing
0:17:32.554761906  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.380952380, resyncing
0:17:34.967663226  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.010839002, resyncing
0:17:35.431429896  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.000068027, resyncing
0:17:35.894114840  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.010861678, resyncing
0:17:36.637802804  3698 0x7fe08806d800 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<alsasink0> Unexpected discontinuity in audio timestamps of +0:00:00.380929705, resyncing

I have tried -as pulsesink, the situation with screen mirroring is even worse. Now, I don't only get these: 0:00:42.706016524 3812 0x7f5b7c002520 WARN audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<pulsesink0> Unexpected discontinuity in audio timestamps of +0:00:00.000090702, resyncing but also a huge number of these messages:

0:00:42.431157080  3812 0x7f5b60200840 WARN                   pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<pulsesink0> Got underflow
0:00:42.614454593  3812 0x7f5b60200840 WARN                   pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<pulsesink0> Got underflow
0:00:42.687117081  3812 0x7f5b60200840 WARN                   pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<pulsesink0> Got underflow

With PulseAudio and ALAC mode, the issue at the beginning of the track is still happening almost always: 0:05:25.952175024 3812 0x7f5b7c002520 WARN audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<pulsesink1> Unexpected discontinuity in audio timestamps of +0:00:00.662517006, resyncing Sometimes even in pure audio mode, these also show up during the first moments of the track:

:18:50.491325575  3812 0x7f5b60200870 WARN                   pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<pulsesink1> Got underflow
0:18:50.560076902  3812 0x7f5b60200870 WARN                   pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<pulsesink1> Got underflow
0:18:50.629741233  3812 0x7f5b60200870 WARN                   pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<pulsesink1> Got underflow
0:18:50.675810761  3812 0x7f5b7c002520 WARN           audiobasesink gstaudiobasesink.c:1786:gst_audio_base_sink_get_alignment:<pulsesink1> Unexpected discontinuity in audio timestamps of +0:00:01.803900226, resyncing

After some play, it seems that at least there are no issues in the middle of the track with PulseAudio sink. I will keep running in PulseAudio mode to see if they still happen.

Hope this sheds some light on what is going on.

arampak commented 8 months ago

To add to this, when using Windows 10 on the same machine with the same hardware and running UxPlay I have no issue at all: everything is smooth and nice in all the modes, both audio and video.

arampak commented 8 months ago

OK, the case can be closed. I changed the USB Wifi card, which immediately solved the problem. I will look into the issues with the Linux driver for the previous card, but the problem was definitely related to the problems with the network card. Sorry for the trouble.

thiccaxe commented 8 months ago

What is your network card?

arampak commented 8 months ago

What is your network card?

It's a no-name USB plug bt+wifi 2.4+5GHz combo that I have received along with the purchase of the refurbished computer from Amazon. When plugged into my FreeBSD notebook, pciconf yealds this:

re0@pci0:2:0:0: class=0x020000 rev=0x10 hdr=0x00 vendor=0x10ec device=0x8168 subvendor=0x17aa subdevice=0x3821
    vendor     = 'Realtek Semiconductor Co., Ltd.'
    device     = 'RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller'
    class      = network
    subclass   = ethernet

It seems to be a Chinese clone with presents itself as Realtec-compatible. lsusb -v provides this:

Bus /dev/usb Device /dev/ugen0.5: ID 0bda:c820 Realtek Semiconductor Corp. 
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 ?
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x0bda Realtek Semiconductor Corp.
  idProduct          0xc820 
  bcdDevice            2.00
  iManufacturer           1 Realtek
  iProduct                2 802.11ac NIC
  iSerial                 3 123456
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength          229
    bNumInterfaces          3
    bConfigurationValue     1
...

followed by a long list of interface and endpoint descriptors. The serial number reported by the device points to its core chip not coming from the original manufacturer, but I may be wrong. I have also noticed that Linux would lose connection to the wifi network after being suspended/resumed, and I had to remove and re-insert the plug to make it work again.

It comes with a small CD containing a Windows driver, but Windows 10 supports it natively, and none of the problems I have had with it in Linux manifested themselves in Windows. It worked quite smoothly, even after Windows was suspended and resumed. So it could be a driver/hardware communication issue.

arampak commented 8 months ago

Interestingly, when I Google the particular set of vendor/subvendor codes, the device seems to be actually shipped by Lenovo (the mini-PC is also branded as Lenovo). So, while it does not carry the Lenovo brand on the packaging, it is not totally no-name. So it seems to be Lenovo supplied Realtek RTL8168B combo device. It's rather popular, it seems.

fduncanh commented 8 months ago

seems to be resolved