search

FDH2 / UxPlay

AirPlay Unix mirroring server
GNU General Public License v3.0
1.35k stars 72 forks source link

installed on RPI 4 server open connection but no mirroring #63

Closed swingstate closed 2 years ago

swingstate commented 2 years ago

cloned latest build, then build and installed on a PI4 w/ 2GB on ubuntu server 21.10 impish

Build went fine, no errors Server starts and accepts connections from local Macbook (12.3 beta) and iPhone 12 (IOS 15.4)

output in terminal looks fine: 

pi@ubuntu:~/UxPlay$ ./uxplay
using system MAC address dc:a6:32:01:a2:fb
Initialized server socket(s)
Accepted IPv4 client on socket 27
Local: 192.168.178.105
Remote: 192.168.178.37
Open connections: 1
Client identified as User-Agent: AirPlay/610.14.1
Accepted IPv4 client on socket 29
Local: 192.168.178.105
Remote: 192.168.178.37
Open connections: 2
raop_rtp_mirror starting mirroring
raop_rtp_mirror tcp socket closed
Connection closed for socket 27
Destroying connection
Open connections: 1
Connection closed for socket 29
Destroying connection
Open connections: 0

PI is connected through HDMI to 34" screen. Although everything looks fine no content is mirrored.

Tried through SSH and with keyboard in front of screen, same results.

RPIplay works on some server, though audio does not mirror hence wanted to use UxPlay.

Any advise?

fduncanh commented 2 years ago

To my knowledge, UxPlay has not been tested on RPi4, but the RPi4 should be powerful enough for gstreamer to work OK.

To see what is going on with your setup.

(1) run uxplay -d to get explicit (RAOP) debug output. (2) GStreamer can be debugged using "export GST_DEBUG=2" before running uxplay (3) repeat , using GST_DEBUG=4

Hopefully, one would be able to see what your issue is from these

swingstate commented 2 years ago

To me the log looks fine from what I understand. Tried again from my Macbook and there seems to be a Videostream. However, the picture on the screen does not change, I only see the ubuntu console.

CPU load is 15 - 20%, few peaks at 45% when running uxplay Memory load is 130MB

I attached the log file.

uxplay-pi4-log.txt

fduncanh commented 2 years ago

Video was streaming, but not being displayed.

Then there is a TEARDOWN request. I suppose this is you stopping the client (clicking on "StopStreaming"?) Or did it stop by itself?

I suspect you dont have a working GStreamer pipeline. probably you have not installed a needed gstreamer plugin (?)

the uxplay debug system doesnt tell you about GStreamer. GStreamer's debug is activated by "export GST_DEBUG=2" in the terminal window, before you run uxplay. Then move up to GST_DEBUG=4

(which will have too much info, I'm afraid.)

raop_rtp_mirror video ntp = 1644531871848795, now = 1644531871783887, latency = -64908
raop_rtp_mirror video ntp = 1644531871882128, now = 1644531871817616, latency = -64512
raop_rtp_mirror video ntp = 1644531871915461, now = 1644531871850534, latency = -64927
raop_rtp_mirror video ntp = 1644531871948795, now = 1644531871883648, latency = -65147
raop_rtp_mirror video ntp = 1644531871982128, now = 1644531871916594, latency = -65534
raop_rtp_mirror video ntp = 1644531872015461, now = 1644531871951556, latency = -63905
raop_rtp_mirror video ntp = 1644531872048795, now = 1644531871983551, latency = -65244

Received video streaming performance info packet from client
raop_rtp_mirror tcp socket closed
httpd receiving on socket 27
conn_request

TEARDOWN rtsp://192.168.178.105/7733720095892062347 RTSP/1.0
Content-Length: 69
Content-Type: application/x-apple-binary-plist
CSeq: 18
DACP-ID: 7AE31B99A69BFEDD
Active-Remote: 3429634580
User-Agent: AirPlay/610.14.1
fduncanh commented 2 years ago

GST_DEBUG=2 only reports errors

GST_DEBUG=4 reports EVERYTHING!

swingstate commented 2 years ago

not sure if I got your advise 100%, but this is with "2", sounds a bit like it cannot connect to the PIs HDMI? And yes, I stopped mirroring which was logged rightfully.


pi@ubuntu:~$ export GST_DEBUG=2
pi@ubuntu:~$ uxplay
0:00:00.265922064  1393  0x14b2e00 WARN                gleglgbm gstgl_gbm_utils.c:473:gst_gl_gbm_find_and_open_drm_node: Found no matching DRM devices
0:00:00.266035859  1393  0x14b2e00 ERROR              gldisplay gstgldisplay_gbm.c:439:gst_gl_display_gbm_new: could not find or open DRM device
0:00:00.269611998  1393  0x14b2e00 WARN                glwindow gstglwindow.c:323:gst_gl_window_new: Could not create window. user specified (null), creating dummy window
0:00:00.270202957  1393  0x1477928 WARN               glcontext gstglcontext.c:1257:gst_gl_context_create_thread:<glcontextegl0> Failed to create context
0:00:00.270336456  1393  0x14b2e00 WARN             glimagesink gstglimagesink.c:1095:_ensure_gl_setup:<sink> error: Failed to initialize egl: EGL_NOT_INITIALIZED
0:00:01.696920602  1393  0x14b2e00 WARN                 kmssink gstkmssink.c:848:gst_kms_sink_start:<video_sink-actual-sink-kms> error: Could not open DRM module (NULL)
0:00:01.696981749  1393  0x14b2e00 WARN                 kmssink gstkmssink.c:848:gst_kms_sink_start:<video_sink-actual-sink-kms> error: reason: No such file or directory (2)
0:00:01.697100545  1393  0x14b2e00 WARN                basesink gstbasesink.c:5869:gst_base_sink_change_state:<video_sink-actual-sink-kms> error: Failed to start
0:00:01.699453397  1393  0x14b2e00 WARN              ximagesink ximagesink.c:867:gst_x_image_sink_xcontext_get:<video_sink-actual-sink-ximage> error: Could not initialise X output
0:00:01.699624007  1393  0x14b2e00 WARN              ximagesink ximagesink.c:867:gst_x_image_sink_xcontext_get:<video_sink-actual-sink-ximage> error: Could not open display
0:00:01.702852908  1393  0x14b2e00 WARN             waylandsink gstwaylandsink.c:362:gst_wayland_sink_find_display:<video_sink-actual-sink-wayland> warning: Could not initialise Wayland output
0:00:01.702913704  1393  0x14b2e00 WARN             waylandsink gstwaylandsink.c:362:gst_wayland_sink_find_display:<video_sink-actual-sink-wayland> warning: Failed to create GstWlDisplay: 'Failed to connect to the wayland display '(default)''

When I start screen mirroring:

Initialized server socket(s)
Accepted IPv4 client on socket 27
Local: 192.168.178.105
Remote: 192.168.178.37
Open connections: 1
Client identified as User-Agent: AirPlay/610.14.1
Accepted IPv4 client on socket 29
Local: 192.168.178.105
Remote: 192.168.178.37
Open connections: 2
raop_rtp_mirror starting mirroring
0:04:02.122967268  1393  0x1477960 WARN            v4l2videodec gstv4l2videodec.c:861:gst_v4l2_video_dec_decide_allocation:<v4l2h264dec0> Duration invalid, not setting latency
0:04:02.294539641  1393  0x1477f88 WARN          v4l2bufferpool gstv4l2bufferpool.c:1288:gst_v4l2_buffer_pool_dqbuf:<v4l2h264dec0:pool0:src> Driver should never set v4l2_buffer.field to ANY
^CStopping...
Removing connection for socket 27
Destroying connection
Open connections: 1
Removing connection for socket 29
Destroying connection
Open connections: 0
swingstate commented 2 years ago

and log level 4, attached due to size.

Log says "...Could not initialise Wayland output..." . That's beyond my knowledge, however I'll google that later and try to understand if that's a plugin or lib I need to install.

uxplay-pi4-loglevel-4.txt

fduncanh commented 2 years ago

gstreamer tried all the videosinks it could find to test if they worked. none did, (the last one tried was waylandsink for a Wayland system (the intended replacement for X11))

So in the end it created the "fakesink" video sink, something like sending the video stream to die in /dev/null (nowhere)!

:00:01.728554121  1427  0x14a6000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:360:gst_element_factory_create: creating element "fakesink" named "fake-video-sink"

Please at least see if the audio stream works. If the client is a macbook have some youtube video with audio stream.

To see what you might be missing, what do you get with

sudo apt list -- installed | grep -i gstreamer
fduncanh commented 2 years ago

What video hardware does your Rpi 4 use?

fduncanh commented 2 years ago

maybe this is helpful?

https://qengineering.eu/install-gstreamer-1.18-on-raspberry-pi-4.html

fduncanh commented 2 years ago

If you get it working well on RPi4, I'll mention RPi 4 on the README for UxPlay.

I think UxPlay ought to work fine on the 4, but the earlier Pi's are not powerful enough for running GStreamer without a lot of latency.

swingstate commented 2 years ago

What video hardware does your Rpi 4 use?

https://www.raspberrypi.com/products/raspberry-pi-4-model-b/specifications/

The GPU is this:

https://www.cpu-monkey.com/en/igpu-broadcom_videocore_vi-221

swingstate commented 2 years ago

I installed the GStreamer packages by following the page you posted above.

$ sudo apt-get install libx264-dev libjpeg-dev

$ sudo apt-get install libgstreamer1.0-dev \
     libgstreamer-plugins-base1.0-dev \
     libgstreamer-plugins-bad1.0-dev \
     gstreamer1.0-plugins-ugly \
     gstreamer1.0-tools \
     gstreamer1.0-gl \
     gstreamer1.0-gtk3

$ sudo apt-get install gstreamer1.0-pulseaudio

No changes. I can pick Uxplay as Airplay target on the IPhone and connect to it but nothing is displayed or audible from the screen. If I listen to music and switch to Airplay the stream is still running in the IPhone app but no audio is coming from the Screen.

Short logs from audio streaming test:

uxplay
using system MAC address dc:a6:32:01:a2:fb
Initialized server socket(s)
Accepted IPv4 client on socket 27
Local: 192.168.178.105
Remote: 192.168.178.102
Open connections: 1
Client identified as User-Agent: AirPlay/610.14.21
Accepted IPv4 client on socket 29
Local: 192.168.178.105
Remote: 192.168.178.102
Open connections: 2
ct=2 spf=352 usingScreen=0 isMedia=1  audioFormat=0x40000
start audio connection, format ALAC 44100/16/2
raop_rtp starting audio
AL lib: (EE) ALCplaybackAlsa_reset: snd_pcm_hw_params(self->pcmHandle, hp) failed: File descriptor in bad stateConnection closed for socket 29
Destroying connection
Open connections: 1
Connection closed for socket 27
Destroying connection
Open connections: 0
^CStopping...

Full logs: uxplay-loglevel4-audiostream.txt

fduncanh commented 2 years ago

in the above debug,

GStreamer cannot find either working audio or working video on your system. It gives up and creates "fakesinks" for both. according to the debug log.

This isnt an issue in UxPlay. Try testing gstreamer by itself and get it working before trying uxplay.

what exactly does

sudo apt list -- installed | grep -i gstreamer
show?

maybe its a hardware issue? some kind of configuration needed? Focus on getting GStreamer properly installed and working. When that is done, uxplay should work

line 164

0:00:00.228573648  4620  0x223f000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:360:gst_element_factory_create: creating element "fakesink" named "fake-audio-sink"

line 673

0:00:01.728988999  4620  0x223f000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:360:gst_element_factory_create: creating element "fakesink" named "fake-video-sink"
fduncanh commented 2 years ago

Your video will use OpenGL. (-vs glimagesink) It's got hardware decoding for H264 video so should work just great with UxPlay when you have it properly set up. I guess google will be your friend in this..... There must be a good RPi 4 community.

H.265 (4kp60 decode), H264 (1080p60 decode, 1080p30 encode)
OpenGL ES 3.1, Vulkan 1.0

Probably got good audio too.

fduncanh commented 2 years ago

for audio the article I referenced says

$ sudo apt-get install gstreamer1.0-pulseaudio

did you do this?

First get audio working, then work on the video. EDIT OK I see you did this

fduncanh commented 2 years ago

I assume you did do the suggested test?

With all GStreamer modules installed let's test the installation with $ gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink .
fduncanh commented 2 years ago

Note that If any special item in the GStreamer pipeline turns out to be needed for this hardware it would be easy to make a "-rpi4" option to add it to uxplays's gstreamer pipelines.

UxPlay is mainly focussed on what happens before the video and audio are handed over to gstreamer.

what happens after that is a kind of "black box"

swingstate commented 2 years ago

I bet this is a config issue, and not hardware related. 

sudo apt list --installed | grep -i gstreamer

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

gir1.2-gstreamer-1.0/impish,now 1.18.5-1 armhf [installed,automatic]
gstreamer1.0-gl/impish,now 1.18.5-1 armhf [installed]
gstreamer1.0-gtk3/impish,now 1.18.5-1ubuntu2 armhf [installed]
gstreamer1.0-libav/impish,now 1.18.5-1 armhf [installed]
gstreamer1.0-plugins-bad/impish,now 1.18.5-1ubuntu1 armhf [installed]
gstreamer1.0-plugins-base/impish,now 1.18.5-1 armhf [installed,automatic]
gstreamer1.0-plugins-good/impish,now 1.18.5-1ubuntu2 armhf [installed,automatic]
gstreamer1.0-plugins-ugly/impish,now 1.18.5-1 armhf [installed]
gstreamer1.0-pulseaudio/impish,now 1.18.5-1ubuntu2 armhf [installed]
gstreamer1.0-tools/impish,now 1.18.5-1 armhf [installed]
gstreamer1.0-vaapi/impish,now 1.18.5-1 armhf [installed]
gstreamer1.0-x/impish,now 1.18.5-1 armhf [installed,automatic]
libgstreamer-gl1.0-0/impish,now 1.18.5-1 armhf [installed,automatic]
libgstreamer-opencv1.0-0/impish,now 1.18.5-1ubuntu1 armhf [installed,automatic]
libgstreamer-plugins-bad1.0-0/impish,now 1.18.5-1ubuntu1 armhf [installed,automatic]
libgstreamer-plugins-bad1.0-dev/impish,now 1.18.5-1ubuntu1 armhf [installed]
libgstreamer-plugins-base1.0-0/impish,now 1.18.5-1 armhf [installed,automatic]
libgstreamer-plugins-base1.0-dev/impish,now 1.18.5-1 armhf [installed]
libgstreamer-plugins-good1.0-0/impish,now 1.18.5-1ubuntu2 armhf [installed,automatic]
libgstreamer-plugins-good1.0-dev/impish,now 1.18.5-1ubuntu2 armhf [installed,automatic]
libgstreamer1.0-0/impish,now 1.18.5-1 armhf [installed,automatic]
libgstreamer1.0-dev/impish,now 1.18.5-1 armhf [installed]

I'll spend some time tonight to troubleshoot further and go through your comments.

fduncanh commented 2 years ago

first thing, do the gstreamer test:

gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

swingstate commented 2 years ago

I did a fresh install with RPI OS 64Bit, installed the packages and can do the streamer test on the RPI terminal (not remotely through SSH). I can start uxplay and from the terminal it looks like it its accepting connections, however no audio or video makes it to the screen.

fduncanh commented 2 years ago

The audio and video is indeed making it through to gstreamer, but being dumped into the "fakesinks" because gstreamer cannnot find where they should go.

If the test

gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink

works, looking to see what it is doing right with GST_DEBUG=4 (start with =2) might help see what working videosink was found

fduncanh commented 2 years ago

It seems I am wrong about the "fakesink" being a problem. I am now comparing your unsuccessful GST_DEBUG=4 trace with what I get for a working one on ubuntu21.10 I have them too, it seems they are a placeholder for the real sink that will later be chosen.

I'm continuing to compare. I'll update this if I see anything.

fduncanh commented 2 years ago

I compared my and your output. No difference for audio. No failures there

I dont understand why you dont hear audio. were you actually streaming audio as well as video?

Below I marked where you fail you are correctly trying to use glimagesink for video. It seems to get set up but then errors start connected to EGL and wayland (??) https://en.wikipedia.org/wiki/EGL_(API)

this link says 

The [Raspberry Pi](https://en.wikipedia.org/wiki/Raspberry_Pi) single-board computer has an EGL interface to hardware-accelerated 3D graphics rendering.[[11]](https://en.wikipedia.org/wiki/EGL_(API)#cite_note-11)

so some extra stuff for EGL needs to be installed, I guess.

Perhaps you need to be running a Wayland display system on RPi 4B with ubuntu 21.10? try googling for clues https://forums.raspberrypi.com/viewtopic.php?t=313685

But this above says you need the RPi 4 with 4GB not 2GB ....


:00:00.266212735  1427  0x14a6000 INFO      GST_PLUGIN_LOADING gstplugin.c:915:_priv_gst_plugin_load_file_for_registry: plugin "/usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstopengl.so" loaded
0:00:00.266280828  1427  0x14a6000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:360:gst_element_factory_create: creating element "glimagesink" named "video_sink-actual-sink-glimage"
0:00:00.267162099  1427  0x14a6000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:363:gst_element_factory_create: creating element "glupload"
0:00:00.267559355  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseTransform@0x172c0d0> adding pad 'sink'
0:00:00.267610670  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseTransform@0x172c0d0> adding pad 'src'
0:00:00.267650521  1427  0x14a6000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:363:gst_element_factory_create: creating element "glcolorconvert"
0:00:00.267918519  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseTransform@0x172c338> adding pad 'sink'
0:00:00.267965389  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseTransform@0x172c338> adding pad 'src'
0:00:00.268004389  1427  0x14a6000 INFO     GST_ELEMENT_FACTORY gstelementfactory.c:363:gst_element_factory_create: creating element "glcolorbalance"
0:00:00.268322979  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseTransform@0x17340b8> adding pad 'sink'
0:00:00.268371220  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseTransform@0x17340b8> adding pad 'src'
0:00:00.268474663  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstutils.c:1816:gst_element_link_pads_full: trying to link element gluploadelement0:src to element glcolorconvertelement0:sink
0:00:00.268506700  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad gluploadelement0:src
0:00:00.268533737  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad glcolorconvertelement0:sink
0:00:00.268560755  1427  0x14a6000 INFO                GST_PADS gstutils.c:1632:prepare_link_maybe_ghosting: gluploadelement0 and glcolorconvertelement0 in same bin, no need for ghost pads
0:00:00.268598570  1427  0x14a6000 INFO                GST_PADS gstpad.c:2382:gst_pad_link_prepare: trying to link gluploadelement0:src and glcolorconvertelement0:sink
0:00:00.268634607  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<gluploadelement0:sink> pad has no peer
0:00:00.272554226  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<glcolorconvertelement0:src> pad has no peer
0:00:00.272889279  1427  0x14a6000 INFO                GST_PADS gstpad.c:2590:gst_pad_link_full: linked gluploadelement0:src and glcolorconvertelement0:sink, successful
0:00:00.272917816  1427  0x14a6000 INFO               GST_EVENT gstevent.c:1610:gst_event_new_reconfigure: creating reconfigure event
0:00:00.272944908  1427  0x14a6000 INFO               GST_EVENT gstpad.c:5903:gst_pad_send_event_unchecked:<gluploadelement0:src> Received event on flushing pad. Discarding
0:00:00.272986241  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstutils.c:1816:gst_element_link_pads_full: trying to link element glcolorconvertelement0:src to element glcolorbalance0:sink
0:00:00.273015723  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad glcolorconvertelement0:src
0:00:00.273040760  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad glcolorbalance0:sink
0:00:00.273066296  1427  0x14a6000 INFO                GST_PADS gstutils.c:1632:prepare_link_maybe_ghosting: glcolorconvertelement0 and glcolorbalance0 in same bin, no need for ghost pads
0:00:00.273099592  1427  0x14a6000 INFO                GST_PADS gstpad.c:2382:gst_pad_link_prepare: trying to link glcolorconvertelement0:src and glcolorbalance0:sink
0:00:00.273131722  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<gluploadelement0:sink> pad has no peer
0:00:00.277023249  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<glcolorbalance0:src> pad has no peer
0:00:00.277139933  1427  0x14a6000 INFO                GST_PADS gstpad.c:2590:gst_pad_link_full: linked glcolorconvertelement0:src and glcolorbalance0:sink, successful
0:00:00.277164989  1427  0x14a6000 INFO               GST_EVENT gstevent.c:1610:gst_event_new_reconfigure: creating reconfigure event
0:00:00.277196896  1427  0x14a6000 INFO               GST_EVENT gstpad.c:5903:gst_pad_send_event_unchecked:<glcolorconvertelement0:src> Received event on flushing pad. Discarding
0:00:00.277235895  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad gluploadelement0:sink
0:00:00.277306099  1427  0x14a6000 INFO                GST_PADS gstpad.c:2382:gst_pad_link_prepare: trying to link sink:proxypad6 and gluploadelement0:sink
0:00:00.277334636  1427  0x14a6000 INFO                GST_PADS gstpad.c:2590:gst_pad_link_full: linked sink:proxypad6 and gluploadelement0:sink, successful
0:00:00.277360043  1427  0x14a6000 INFO               GST_EVENT gstevent.c:1610:gst_event_new_reconfigure: creating reconfigure event
0:00:00.277396654  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstGLSinkBin@0x1727078> adding pad 'sink'
0:00:00.277469023  1427  0x14a6000 INFO       gstcontrolbinding gstcontrolbinding.c:139:gst_control_binding_constructor:<glcolorbalance0> trying to put property 'contrast' under control
0:00:00.277518042  1427  0x14a6000 INFO       gstcontrolbinding gstcontrolbinding.c:139:gst_control_binding_constructor:<glcolorbalance0> trying to put property 'brightness' under control
0:00:00.277558708  1427  0x14a6000 INFO       gstcontrolbinding gstcontrolbinding.c:139:gst_control_binding_constructor:<glcolorbalance0> trying to put property 'hue' under control
0:00:00.277599522  1427  0x14a6000 INFO       gstcontrolbinding gstcontrolbinding.c:139:gst_control_binding_constructor:<glcolorbalance0> trying to put property 'saturation' under control
0:00:00.277886983  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:752:gst_element_add_pad:<GstBaseSink@0x173b928> adding pad 'sink'
0:00:00.277972612  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstutils.c:1816:gst_element_link_pads_full: trying to link element glcolorbalance0:src to element sink:sink
0:00:00.278005908  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad glcolorbalance0:src
0:00:00.278032167  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad sink:sink
0:00:00.278059575  1427  0x14a6000 INFO                GST_PADS gstutils.c:1632:prepare_link_maybe_ghosting: glcolorbalance0 and sink in same bin, no need for ghost pads
0:00:00.278095852  1427  0x14a6000 INFO                GST_PADS gstpad.c:2382:gst_pad_link_prepare: trying to link glcolorbalance0:src and sink:sink
0:00:00.278139537  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<(NULL):sink> pad has no peer
0:00:00.282174285  1427  0x14a6000 INFO                GST_PADS gstpad.c:2590:gst_pad_link_full: linked glcolorbalance0:src and sink:sink, successful
0:00:00.282209118  1427  0x14a6000 INFO               GST_EVENT gstevent.c:1610:gst_event_new_reconfigure: creating reconfigure event
0:00:00.282235748  1427  0x14a6000 INFO               GST_EVENT gstpad.c:5903:gst_pad_send_event_unchecked:<glcolorbalance0:src> Received event on flushing pad. Discarding
0:00:00.282297358  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:1009:gst_element_get_static_pad: found pad video_sink-actual-sink-glimage:sink
0:00:00.284378454  1427  0x14a6000 INFO              GST_STATES gstbin.c:2528:gst_bin_element_set_state:<sink> current NULL pending VOID_PENDING, desired next READY
0:00:00.284452454  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<video_sink-actual-sink-glimage:sink> pad has no peer
0:00:00.284483083  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:115:pad_query:<sink:sink> pad peer query failed
0:00:00.284507583  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:176:_gst_context_query:<sink> posting need context message
0:00:00.284569064  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<video_sink-actual-sink-glimage:sink> pad has no peer
0:00:00.284594064  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:115:pad_query:<sink:sink> pad peer query failed
0:00:00.284617841  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:176:_gst_context_query:<sink> posting need context message
0:00:00.284654156  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<video_sink-actual-sink-glimage:sink> pad has no peer
0:00:00.284681415  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:115:pad_query:<sink:sink> pad peer query failed
0:00:00.284704878  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:176:_gst_context_query:<sink> posting need context message
0:00:00.284747433  1427  0x14a6000 INFO               gldisplay gstgldisplay.c:290:gst_gl_display_new: creating a display, user choice:(NULL) (platform: (NULL))
0:00:00.285507279  1427  0x14a6000 INFO               gldisplay gstgldisplay_wayland.c:113:gst_gl_display_wayland_new: Failed to open Wayland display connection.
0:00:00.286008498  1427  0x14a6000 INFO               gldisplay gstgldisplay_x11.c:109:gst_gl_display_x11_new: Failed to open X11 display connection with name, '(null)'

****errors start here!****

0:00:00.286860621  1427  0x14a6000 WARN                gleglgbm gstgl_gbm_utils.c:473:gst_gl_gbm_find_and_open_drm_node: Found no matching DRM devices
0:00:00.286944861  1427  0x14a6000 ERROR              gldisplay gstgldisplay_gbm.c:439:gst_gl_display_gbm_new: could not find or open DRM device
0:00:00.290485113  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:240:gst_gl_element_propagate_display_context:<sink> posting have context (0x1744a00) message with display (0x1742850)
0:00:00.290573594  1427  0x14a6000 INFO                GST_PADS gstpad.c:4314:gst_pad_peer_query:<video_sink-actual-sink-glimage:sink> pad has no peer
0:00:00.290607038  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:115:pad_query:<sink:sink> pad peer query failed
0:00:00.290631057  1427  0x14a6000 INFO             GST_CONTEXT gstglutils.c:176:_gst_context_query:<sink> posting need context message
0:00:00.290702223  1427  0x14a6000 INFO               glcontext gstglcontext.c:337:gst_gl_context_new: creating a context for display <gldisplayegl0>, user choice:(null)
0:00:00.290799722  1427  0x14a6000 INFO                glwindow gstglwindow.c:278:gst_gl_window_new: creating a window, user choice:(null)
0:00:00.290828907  1427  0x14a6000 INFO                glwindow gstglwindow_x11.c:136:gst_gl_window_x11_new: Wrong display type 32 for this window type 1
0:00:00.290854111  1427  0x14a6000 WARN                glwindow gstglwindow.c:323:gst_gl_window_new: Could not create window. user specified (null), creating dummy window
0:00:00.291213145  1427  0x1704528 INFO               glcontext gstglcontext.c:1250:gst_gl_context_create_thread:<glcontextegl0> Attempting to create opengl context. user chosen api(s) (any), compiled api support (opengl opengl3 gles2) display api (opengl opengl3 gles2)
0:00:00.291409088  1427  0x1704528 WARN               glcontext gstglcontext.c:1257:gst_gl_context_create_thread:<glcontextegl0> Failed to create context
0:00:00.291460439  1427  0x14a6000 INFO               glcontext gstglcontext.c:1060:gst_gl_context_create:<glcontextegl0> gl thread created
0:00:00.291542402  1427  0x14a6000 WARN             glimagesink gstglimagesink.c:1095:_ensure_gl_setup:<sink> error: Failed to initialize egl: EGL_NOT_INITIALIZED
0:00:00.291582309  1427  0x14a6000 INFO        GST_ERROR_SYSTEM gstelement.c:2234:gst_element_message_full_with_details:<sink> posting message: Failed to initialize egl: EGL_NOT_INITIALIZED
0:00:00.291627827  1427  0x14a6000 INFO        GST_ERROR_SYSTEM gstelement.c:2261:gst_element_message_full_with_details:<sink> posted error message: Failed to initialize egl: EGL_NOT_INITIALIZED
0:00:00.291656975  1427  0x14a6000 INFO              GST_STATES gstelement.c:3054:gst_element_change_state:<sink> have FAILURE change_state return
0:00:00.291685197  1427  0x14a6000 INFO              GST_STATES gstelement.c:2641:gst_element_abort_state:<sink> aborting state from NULL to READY
0:00:00.291717234  1427  0x14a6000 INFO              GST_STATES gstbin.c:2993:gst_bin_change_state_func:<video_sink-actual-sink-glimage> child 'sink' failed to go to state 2(READY)
0:00:00.291765511  1427  0x14a6000 INFO              GST_STATES gstelement.c:2769:gst_element_continue_state:<sink> completed state change to NULL
0:00:00.291798789  1427  0x14a6000 INFO              GST_STATES gstelement.c:2769:gst_element_continue_state:<glcolorbalance0> completed state change to NULL
0:00:00.291831529  1427  0x14a6000 INFO              GST_STATES gstelement.c:2769:gst_element_continue_state:<glcolorconvertelement0> completed state change to NULL
0:00:00.291862103  1427  0x14a6000 INFO              GST_STATES gstelement.c:2769:gst_element_continue_state:<gluploadelement0> completed state change to NULL
0:00:00.291898084  1427  0x14a6000 INFO              GST_STATES gstelement.c:3054:gst_element_change_state:<video_sink-actual-sink-glimage> have FAILURE change_state return
0:00:00.291967139  1427  0x14a6000 INFO              GST_STATES gstbin.c:2528:gst_bin_element_set_state:<sink> current NULL pending VOID_PENDING, desired next NULL
0:00:00.291994973  1427  0x14a6000 INFO              GST_STATES gstbin.c:2646:gst_bin_element_set_state:<sink> skipping transition from NULL to  NULL
0:00:00.292022454  1427  0x14a6000 INFO              GST_STATES gstbin.c:2977:gst_bin_change_state_func:<video_sink-actual-sink-glimage> child 'sink' changed state to 1(NULL) successfully
0:00:00.292053528  1427  0x14a6000 INFO              GST_STATES gstbin.c:2528:gst_bin_element_set_state:<glcolorbalance0> current NULL pending VOID_PENDING, desired next NULL
0:00:00.292080639  1427  0x14a6000 INFO              GST_STATES gstbin.c:2646:gst_bin_element_set_state:<glcolorbalance0> skipping transition from NULL to  NULL
0:00:00.292107953  1427  0x14a6000 INFO              GST_STATES gstbin.c:2977:gst_bin_change_state_func:<video_sink-actual-sink-glimage> child 'glcolorbalance0' changed state to 1(NULL) successfully
0:00:00.292139101  1427  0x14a6000 INFO              GST_STATES gstbin.c:2528:gst_bin_element_set_state:<glcolorconvertelement0> current NULL pending VOID_PENDING, desired next NULL
0:00:00.292165101  1427  0x14a6000 INFO              GST_STATES gstbin.c:2646:gst_bin_element_set_state:<glcolorconvertelement0> skipping transition from NULL to  NULL
0:00:00.292192230  1427  0x14a6000 INFO              GST_STATES gstbin.c:2977:gst_bin_change_state_func:<video_sink-actual-sink-glimage> child 'glcolorconvertelement0' changed state to 1(NULL) successfully
0:00:00.292222119  1427  0x14a6000 INFO              GST_STATES gstbin.c:2528:gst_bin_element_set_state:<gluploadelement0> current NULL pending VOID_PENDING, desired next NULL
0:00:00.292247878  1427  0x14a6000 INFO              GST_STATES gstbin.c:2646:gst_bin_element_set_state:<gluploadelement0> skipping transition from NULL to  NULL
0:00:00.292275934  1427  0x14a6000 INFO              GST_STATES gstbin.c:2977:gst_bin_change_state_func:<video_sink-actual-sink-glimage> child 'gluploadelement0' changed state to 1(NULL) successfully
0:00:00.292303878  1427  0x14a6000 INFO              GST_STATES gstelement.c:2769:gst_element_continue_state:<video_sink-actual-sink-glimage> completed state change to NULL
0:00:00.292344637  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2137:gst_pad_unlink: unlinking glcolorbalance0:src(0x1714690) and sink:sink(0x1714940)
0:00:00.292383414  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2192:gst_pad_unlink: unlinked glcolorbalance0:src and sink:sink
0:00:00.292417284  1427  0x14a6000 INFO           GST_PARENTAGE gstbin.c:1826:gst_bin_remove_func:<video_sink-actual-sink-glimage> removed child "sink"
0:00:00.292464858  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2137:gst_pad_unlink: unlinking glcolorconvertelement0:src(0x17143e0) and glcolorbalance0:sink(0x1714538)
0:00:00.292499691  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2192:gst_pad_unlink: unlinked glcolorconvertelement0:src and glcolorbalance0:sink
0:00:00.292532783  1427  0x14a6000 INFO           GST_PARENTAGE gstbin.c:1826:gst_bin_remove_func:<video_sink-actual-sink-glimage> removed child "glcolorbalance0"
0:00:00.292567505  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3345:gst_element_dispose:<glcolorbalance0> 0x17340b8 dispose
0:00:00.292593931  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<glcolorbalance0> removing pad 'sink'
0:00:00.292628061  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<glcolorbalance0> removing pad 'src'
0:00:00.292659968  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3391:gst_element_dispose:<glcolorbalance0> 0x17340b8 parent class dispose
0:00:00.292715671  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3423:gst_element_finalize:<glcolorbalance0> 0x17340b8 finalize
0:00:00.292742615  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3428:gst_element_finalize:<glcolorbalance0> 0x17340b8 finalize parent
0:00:00.292777337  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2137:gst_pad_unlink: unlinking gluploadelement0:src(0x1701cc8) and glcolorconvertelement0:sink(0x1714288)
0:00:00.292812244  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2192:gst_pad_unlink: unlinked gluploadelement0:src and glcolorconvertelement0:sink
0:00:00.292845818  1427  0x14a6000 INFO           GST_PARENTAGE gstbin.c:1826:gst_bin_remove_func:<video_sink-actual-sink-glimage> removed child "glcolorconvertelement0"
0:00:00.292878651  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3345:gst_element_dispose:<glcolorconvertelement0> 0x172c338 dispose
0:00:00.292904744  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<glcolorconvertelement0> removing pad 'sink'
0:00:00.292935151  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<glcolorconvertelement0> removing pad 'src'
0:00:00.292966410  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3391:gst_element_dispose:<glcolorconvertelement0> 0x172c338 parent class dispose
0:00:00.292993521  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3423:gst_element_finalize:<glcolorconvertelement0> 0x172c338 finalize
0:00:00.293019335  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3428:gst_element_finalize:<glcolorconvertelement0> 0x172c338 finalize parent
0:00:00.293056539  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2137:gst_pad_unlink: unlinking sink:proxypad6(0x16e9908) and gluploadelement0:sink(0x1714130)
0:00:00.293086946  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstpad.c:2192:gst_pad_unlink: unlinked sink:proxypad6 and gluploadelement0:sink
0:00:00.293114316  1427  0x14a6000 INFO           GST_PARENTAGE gstbin.c:1826:gst_bin_remove_func:<video_sink-actual-sink-glimage> removed child "gluploadelement0"
0:00:00.293146260  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3345:gst_element_dispose:<gluploadelement0> 0x172c0d0 dispose
0:00:00.293171945  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<gluploadelement0> removing pad 'sink'
0:00:00.293203630  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<gluploadelement0> removing pad 'src'
0:00:00.293234630  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3391:gst_element_dispose:<gluploadelement0> 0x172c0d0 parent class dispose
0:00:00.293268611  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3423:gst_element_finalize:<gluploadelement0> 0x172c0d0 finalize
0:00:00.293294130  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3428:gst_element_finalize:<gluploadelement0> 0x172c0d0 finalize parent
0:00:00.293321167  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3345:gst_element_dispose:<video_sink-actual-sink-glimage> 0x1727078 dispose
0:00:00.293346148  1427  0x14a6000 INFO        GST_ELEMENT_PADS gstelement.c:868:gst_element_remove_pad:<video_sink-actual-sink-glimage> removing pad 'sink'
0:00:00.293383222  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3391:gst_element_dispose:<video_sink-actual-sink-glimage> 0x1727078 parent class dispose
0:00:00.293410740  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3423:gst_element_finalize:<video_sink-actual-sink-glimage> 0x1727078 finalize
0:00:00.293436592  1427  0x14a6000 INFO         GST_REFCOUNTING gstelement.c:3428:gst_element_finalize:<video_sink-actual-sink-glimage> 0x1727078 finalize parent
fduncanh commented 2 years ago

It seems that there are a lot of complicated issues right now on RPi 4B, like 32bit vs 64 bit, Debian Bullseye vs Buster, changes in h264 support etc. UxPlay is RPiPlay with improvements and fixes, and gstreamer (usually for X86_64 architecture) instead of the OpenMax renderers that RPiPlay was created to use. (the gstreamer support from UxPlay was backported to RPiPlay, but I think most RPiPlay users use OpenMax.) But now OpenMax is being abandoned as obsolete in Debian-Bullseye-based distros for the Pi, and there are a lot of questions on this at the RPiPlay site.

The RPi 4 seems to have the right hardware (with built in h264 decoding) to run UxPlay well, if you can get gstreamer properly working to use it. If you can get RPiPlay working with its gstreamer option, then you can also run UxPlay. I cant really help with this because I don/t have the RPi 4 , and I now see its not a simple issue. You need to get help from RPi users.

fduncanh commented 2 years ago

I did a fresh install with RPI OS 64Bit, installed the packages and can do the streamer test on the RPI terminal (not remotely through SSH).

This might be because ubuntu-21.10 has moved the default display system from X11 to Wayland. You could try to make your RPi 4 use X11 instead of Wayland to see if that solves things. According to the post below some love Wayland some hate it. https://forums.raspberrypi.com/viewtopic.php?t=313685

I can start uxplay and from the terminal it looks like it its accepting connections, however no audio or video makes it to the screen.

I'm puzzled by the "no audio" as there were no audio errors showing up in the GST_DEBUG. You are sure that audio was streaming, and the volume wasn't turned down? the uxplay -d output will show the audio packets arriving one-by-one among the video packets.

I would say, solve the audio first, then try to solve the video issue.

fduncanh commented 2 years ago

Or the issues could be that you are running Wayland on an RPI 4 with only 2GB memory (?) . Are you running 32 bit or 64 bit ubuntu? I think you need the 32 bit ubuntu for your system. The post says something about things working well only on 4GB or 8GB systems. https://forums.raspberrypi.com/viewtopic.php?t=31368

"If it works on the 400 it will work with the 4B, the only caveat is you will need 4GB or 8GB memory versions, therefore Ubuntu are erring on side of caution, because users do not read anything and would try to install on 1GB & 2GB memory versions."

swingstate commented 2 years ago

appreciate your support! I'll test a few things with the RPI and with VMs to see if this is RPI issue or I am doing a mistake somewhere. Suggest to close the issue.

fduncanh commented 2 years ago

R Pi 4 is now supported. see the readme. (you need to patch GStreamer, until updates come)

fduncanh commented 2 years ago

@swingstate the RPi support is for Bullseye not Buster.