search

FDH2 / UxPlay

AirPlay Unix mirroring server
GNU General Public License v3.0
1.53k stars 78 forks source link

[Question] how to further process stream? #290

Closed gotschi closed 5 months ago

gotschi commented 5 months ago

Thanks for maintaining this project!

I have a question regarding the output as I'm trying to restream it to rtsp:// or rtmp://: The stream out of uxplay is apparently H264 level 3.1 but neither VLC or any other player can play it. -> pps and sps missing I guess, because the saved .h264 file plays fine.

So I'm using this to re-encode the stream on the fly but this just produces a square green screen with some flickering (in the browser, vlc still doesn't display anything):

mkfifo /tmp/stream.h264
uxplay -d -vdmp /tmp/stream  -avdec -s "1280x720@30" \
ffmpeg -i /tmp/stream.h264 -c:v h264_v4l2m2m -pix_fmt yuv420p -refs 4 -f rtsp rtsp://127.0.0.1:8554/hdmi

Using software h264 encoder works but obviously is very slow on a pi zero 2w.

What can I do about this? Is there a known way to achieve this? I'm using go2rtc (for rtsp) and mediamtx (rtmp)

fduncanh commented 5 months ago

It should be possible to stream output with rtsp but uxplay doesn't do this.

The RAOP code in lib decrypts separate H264 video and AAC audio streams from the ios client. These both have timestamps. what would be needed would be a gstreamer pipeline that recombines them into a mp4 container that could be sent on with rtsp. It would be great if you could write the code for that!

gotschi commented 5 months ago

Audio wasn't even relevant here, I was just not expecting the resulting h264 stream to be of some weird format.

It does not really matter as I am building a stream box which takes any hdmi signal and streams it to youtube and/or go2rtc So plugging in the pi hdmi output into the tc358743 which connects to the camera input on the pi makes it work for me, I could probably also output the stream to a loopback display but meh, it works awesome :D Just weird that one cannot reencode this airplay stream directly. thank you

fduncanh commented 5 months ago

The h264 stream is "standard" It starts with a sps/pps NAL and only inserts a new pps/sps NAL when someting changes. If you miss the initial pps/sps (prepended to a video data NAL) it wont work.

I think VLC will expect the h264 to be wrapped into a mp4 container. It would be good to enable uxplay to do this (and include the audio too) but it needs someone experienced to work it out. Then the output could be saved or retransmitted.