Feature Request: RTMP support

[LockonS]

Thanks for this amazing project! I have build and run uxplay on an RPi 3B+ smoothly following the gstreamer patch instruction. I wonder if video stream received by uxplay could be streamed to a rtmp server directly?

[fduncanh]

(nice to get feedback that the video4linux2 patching is being sucessfully used)

my guess is that the video stream could, if you use gstreamer and -vs rtmpsink (I haven't experimented).

https://gstreamer.freedesktop.org/documentation/rtmp/rtmpsink.html?gi-language=c

One issue might be that the audio would need to be recombined with the video, as Airplay sends them separately, and UxPlay processes them separately in different threads. They both come with some timestamps that allow then to be played together simultaneously. (I suspect there might even be some gstreamer way to recombine the video and audio in some mpg4 wrapper)

[LockonS]

Thanks for the reply and the tips about the rtmpsink. I'll look into that and give a try. Since mensioned the success report about the patch on 3B+, there might be some risk to do the patch operation on a 4B running ubuntu 22.04 with GUI. Three times I've tried to get the gstreamer patch done and every time the debuild process failed. Futher more, before I fix the debuild issue, the desktop environment breaks and could never be started again. Reinstall ubuntu-desktop, switch to lightdm, nothing works. The error report from the system journal seems to be a weird one and no solutions found. As a result I did a clean install, each time 😭😭😭. I will post the arranged detail about the incident later and luckily, that might save some others tons of time or at least give a head warning about doing a system backup first.

[fduncanh]

what exactly fails about debuild? The instructions say:

The first step is to download the gstreamer1.0-plugins-good source package: edit /etc/apt/sources.list to uncomment the deb-src entries (or duplicate your deb entries and change "deb" to "deb-src") and then run sudo apt-get update. Then do

sudo apt-get install devscripts sudo apt-get build-dep gstreamer1.0-plugins-good Next (in a directory where you will rebuild the package, and without using "sudo"), do

apt-get source gstreamer1.0-plugins-good cd gst-plugins-good1.0-1.20.1 debuild

This rebuilds the unpatched package. When all has finished, don't worry about a message that "running debsign failed",

next:

patch -p1 < (path to the patch)/v4l2_1_18_4_diff3.txt cd obj*
ninja cd sys/v4l2 strip libgstvideo4linux2.so sudo cp libgstvideo4linux2.so (location)/gstreamer-1.0/

Here

(location) = /usr/lib/aarch64-linux-gnu (64 bit version) (location) = /usr/lib/arm-linux-gnuabihf (32 bit version)

and obj* is either obj-aarch64-linux-gnu or obj-arm-linux-gnueabihf.

The failure of debsign is NOT a failure. You are NOT trying to rebuild the package for installation. You are just creating a starting point for fixing a single file, libgstvideo4linxux2.so There is no way this could mess up your installation, especially if you never got as far as installing the modified version.

Please report full details of what you think is going wrong. And exactly what you did.

I have tested with the standard ubuntu 22.04 not a version with lightdm.

[LockonS]

Not talking about the warnings. I follow the steps and run the debuild process. Plus, the gstreamer version on ubuntu 22.04 running on RPi 4B is 1.20.3. But I think that doesn't matter in this case.

apt list | grep gstreamer
# as can be seen the version is 1.20.3
gstreamer1.0-plugins-good/jammy-updates,now 1.20.3-0ubuntu1 arm64

Actually I want to talk about the problems I encountered and the incident that happend serveral times, which ultimately break the GUI environment in a mysterious way. Apology here as the topic might be a huge stray away from the point.

System image: ubuntu-22.04.1-preinstalled-desktop-arm64+raspi.img.xz (downloaded from ubuntu official site, using gdm3) Hardware: RaspberryPi 4B (4GB RAM)

As a conclusion, take a wild guess here that some package installed in order to get the patch done, breaks the Ubuntu GUI environment running on RPi 4B. Removing all the extra package doesn't fix the problem. I list all the packages installed with apt list --installed before and after the patch routine. After the GUI crashes, I start to uninstall all the extra packages with the two lists compared side by side. That doesn't fix the GUI.

Here is the detail about what happened and the steps I took.

1. Before trying to get the patch done, I build the gstreamer from source following the instruction. Strangely the gstreamer just installed doesn't support v4l2. As the output of gst-inspect-1.0 video4linux2, I didn't see the v4l2h264dec and v4l2h264enc. The complete output is listed here (Luckily I took an copy of the output at that time).

gst-inspect-1.0 video4linux2

Plugin Details:
  Name                     video4linux2
  Description              elements for Video 4 Linux
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
  Version                  1.20.3
  License                  LGPL
  Source module            gst-plugins-good
  Source release date      2022-06-15
  Binary package           GStreamer Good Plugins (Ubuntu)
  Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-good1.0

  v4l2deviceprovider: Video (video4linux2) Device Provider
  v4l2radio: Radio (video4linux2) Tuner
  v4l2sink: Video (video4linux2) Sink
  v4l2src: Video (video4linux2) Source

  4 features:
  +-- 3 elements
  +-- 1 device providers

2. Since this solution was not working, and I've got no idea why, I proceed to retry the patch solution. After a reboot, I start to run the patch routine and the steps I took was like below. Yes, at this time, the GUI environment is working as normal.

# installed a bunch of packages alone with devscripts and some of them might break the GUI
sudo apt-get install devscripts
sudo apt-get build-dep gstreamer1.0-plugins-good
# the first two steps went well, nothing strange happens
apt-get source gstreamer1.0-plugins-good
cd gst-plugins-good1.0-1.20.3
# now get to problem part, the debuild process fails
debuild

3. First time the debuild process failed with a list of unmet dependencies. After got them installed the process went on until another error occurs. A complete list of the dependencies installed in this step is listed here.

sudo apt-get install libavc1394-dev qtbase5-dev libjack-jackd2-dev libcaca-dev libpng-dev libshout3-dev libjpeg-dev libxext-dev libxfixes-dev libxv-dev qtbase5-private-dev qtdeclarative5-dev libqt5x11extras5-dev xvfb libxdamage-dev libjack-jackd2-dev debhelper libcairo2-dev libaa1-dev libgdk-pixbuf2.0-dev libgtk-3-dev libtag1-dev libpulse-dev libmpg123-dev libiec61883-dev

4. The second debuild error occurs when subprocess when ninja process was executing. Sorry I didn't backup the error output, and the searching history was gone with the system reinstall. I thought that debuild fail was no big deal and could be fixed following the error output. But I haven't tried since the GUI has crashed, I thought there is no point trying to get the patch done on that system environment, unless the GUI can be fixed. (Regrets. Should think that through and find the solution first, before I did another system clean install and wipe out everything. I felt like a fool 😭.)

5. As I tried to fix the problem occurs in the debuild process, then I found that GUI freezes with no response. No new windows can be open (Just like the previous time when I broke the GUI environment). Except for the already opened terminal windows. So I just reboot the system, and GUI failed to start as described.

6. After I compare the system journal and xession error log between an working GUI and the failed GUI, I noticed serveral lines of warning and error. And it seems to have something to do with the package libgdk-pixbuf2.0-dev, which got installed in step 3. The error info in $HOME/.xsession-errors also point to the package

# a part of system journal
Aug 27 13:11:56 dreamisty-node org.gnome.OnlineAccounts[1906]: /usr/libexec/goa-daemon: /usr/local/lib/aarch64-linux-gnu/libpng16.so.16: no version information available (required by /lib/aarch64-linux-gnu/libgdk_pixbuf-2.0.so.0)
Aug 27 13:11:56 dreamisty-node org.gnome.OnlineAccounts[1906]: /usr/libexec/goa-daemon: /usr/local/lib/aarch64-linux-gnu/libpng16.so.16: no version information available (required by /lib/aarch64-linux-gnu/libwebkit2gtk-4.0.so.37)
# the error in xsession-errors
/usr/libexec/gnome-session-binary: /usr/local/lib/aarch64-linux-gnu/libpng16.so.16: no version information available (required by /lib/aarch64-linux-gnu/libgdk_pixbuf-2.0.so.0)
/usr/libexec/gnome-session-binary: symbol lookup error: /lib/aarch64-linux-gnu/libgdk-3.so.0: undefined symbol: cairo_xlib_surface_get_display

Hopefully someone might be able to tell us what breaks the GUI environment and how to fix that, or at least prevent some others from wasting time on the same problem if they run Ubuntu on RPi 4B.

[LockonS]

If more detail need to be provided, I'd like to run another routine sometime later and no doubt that will break aother Ubuntu GUI. And please tell me which part I need to pay attention on during the experiment.

[fduncanh]

see #123

[fduncanh]

@LockonS I think your issues may that you have un-updated firmware on your R Pi 4B. check with sudo rpi-eeprom-update. I don't replicate your problem, everything works fine.

If it says there is new firmware, just reboot and it will install.

• I updated the wiki for ubuntu-22.04.1 "jammy update": the debuild "failure" is just in some tests (ignore it), probably because 22.04.1 has a mixture of 1.20.1 base packages and 1.20.3 plugins. The video4linux library builds correctly, and that is all that gets replaced.

EDIT: to make the firmware update at the next reboot, run "sudo rpi-eeprom-update -a", without the "-a" the update doesn't happen.

[fduncanh]

apparently fixed. reopen (or post new issue) if needed