Error on orientation change - RPi

[gregd72002]

UxPlay works fine on my RPi (Bullseye) using the following command: uxplay -v4l2 -bt709 -vs kmssink I can start screen mirroring from my iPhone in portrait or landscape and it works fine. I can see in debug that is uses this pipeline "appsrc name=video_source ! queue ! h264parse ! capssetter caps="video/x-h264, colorimetry=bt709" ! v4l2h264dec ! v4l2convert ! kmssink name=video_sink sync=false" and with

begin video stream wxh = 500x1080; source 500x1080
raop_rtp_mirror width_source = 500.000000 height_source = 1080.000000 width = 500.000000 height = 1080.000000

However when I rotate the device from portrait to landscape (or vice-versa), UxPlay stops streaming with the following error:

pi@rpiplay:~ $ uxplay -v4l2 -bt709 -vs kmssink
using system MAC address b8:27:eb:1f:a9:df
Initialized server socket(s)
Accepted IPv4 client on socket 34
Local: 192.168.1.175
Remote: 192.168.1.189
Open connections: 1
Client identified as User-Agent: AirPlay/620.8.2
Accepted IPv4 client on socket 36
Local: 192.168.1.175
Remote: 192.168.1.189
Open connections: 2
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline

** (uxplay:629): CRITICAL **: 07:35:40.550: gst_v4l2_transform_set_caps: assertion '!GST_V4L2_IS_ACTIVE (self->v4l2output)' failed

** (uxplay:629): CRITICAL **: 07:35:40.557: gst_v4l2_transform_set_caps: assertion '!GST_V4L2_IS_ACTIVE (self->v4l2output)' failed

** (uxplay:629): CRITICAL **: 07:35:40.730: gst_v4l2_transform_set_caps: assertion '!GST_V4L2_IS_ACTIVE (self->v4l2output)' failed
GStreamer error: Internal data stream error.
*** This is a generic GStreamer error that usually means that GStreamer
*** was unable to construct a working video pipeline.
*** If you are letting the default autovideosink select the videosink,
*** GStreamer may be trying to use non-functional hardware h264 video decoding.
*** Try using option -avdec to force software decoding or use -vs <videosink>
*** to select a videosink of your choice (see "man uxplay")
Removing connection for socket 34
Destroying connection
Open connections: 1
raop_rtp_mirror->running is no longer true
Removing connection for socket 36
Destroying connection
Open connections: 0
Initialized server socket(s)

When I look into debug I see the wxh change:

raop_rtp video: now = 1666680297.479539, ntp = 1666680297.582987, latency = -0.103448
raop_rtp_mirror: unidentified extra header data  0.000000, 96.000000
begin video stream wxh = 1920x888; source 1920x888
raop_rtp_mirror width_source = 1920.000000 height_source = 888.000000 width = 1920.000000 height = 888.000000
raop_rtp_mirror: sps/pps header size = 6
raop_rtp_mirror h264 sps/pps header:
01 64 00 28 ff e1 

raop_rtp_mirror sps size = 17
raop_rtp_mirror h264 Sequence Parameter Set:
27 64 00 28 ac 13 14 50 1e 01 c7 e5 9b 80 80 80 
81 

raop_rtp_mirror pps size = 4
raop_rtp_mirror h264 Picture Parameter Set:
28 ee 3c b0 

remainder size = 4
remainder of sps+pps packet:
02 00 00 00 

raop_rtp video: now = 1666680297.603482, ntp = 1666680297.616215, latency = -0.012733
nalu_type = 5, nalu_size = 116896,  processed bytes 116900, payloadsize = 116900 nalus_count = 1
raop_rtp video: now = 1666680297.626447, ntp = 1666680297.699284, latency = -0.072837
raop_rtp video: now = 1666680297.635105, ntp = 1666680297.732512, latency = -0.097407
raop_rtp video: now = 1666680297.667069, ntp = 1666680297.765739, latency = -0.098670
raop_rtp video: now = 1666680297.704804, ntp = 1666680297.798967, latency = -0.094163
raop_rtp video: now = 1666680297.738786, ntp = 1666680297.832195, latency = -0.093409
raop_rtp video: now = 1666680297.768500, ntp = 1666680297.865422, latency = -0.096922
raop_rtp video: now = 1666680297.801423, ntp = 1666680297.898650, latency = -0.097227
raop_rtp video: now = 1666680297.870145, ntp = 1666680297.931983, latency = -0.061838
raop_rtp video: now = 1666680297.899979, ntp = 1666680297.965317, latency = -0.065338

** (uxplay:659): CRITICAL **: 07:44:57.907: gst_v4l2_transform_set_caps: assertion '!GST_V4L2_IS_ACTIVE (self->v4l2output)' failed

** (uxplay:659): CRITICAL **: 07:44:57.913: gst_v4l2_transform_set_caps: assertion '!GST_V4L2_IS_ACTIVE (self->v4l2output)' failed
raop_rtp video: now = 1666680297.955202, ntp = 1666680297.998650, latency = -0.043448
raop_rtp video: now = 1666680297.967437, ntp = 1666680298.031983, latency = -0.064546
raop_rtp video: now = 1666680298.022475, ntp = 1666680298.065317, latency = -0.042842
raop_rtp video: now = 1666680298.049657, ntp = 1666680298.098650, latency = -0.048993

** (uxplay:659): CRITICAL **: 07:44:58.082: gst_v4l2_transform_set_caps: assertion '!GST_V4L2_IS_ACTIVE (self->v4l2output)' failed
raop_rtp video: now = 1666680298.131743, ntp = 1666680298.214313, latency = -0.082570
raop_rtp video: now = 1666680298.151646, ntp = 1666680298.247541, latency = -0.095895
GStreamer error: Internal data stream error.
*** This is a generic GStreamer error that usually means that GStreamer
*** was unable to construct a working video pipeline.
*** If you are letting the default autovideosink select the videosink,
*** GStreamer may be trying to use non-functional hardware h264 video decoding.
*** Try using option -avdec to force software decoding or use -vs <videosink>
*** to select a videosink of your choice (see "man uxplay")
raop_rtp video: now = 1666680298.183251, ntp = 1666680298.280768, latency = -0.097517
raop_rtp video: now = 1666680298.214853, ntp = 1666680298.313996, latency = -0.099143
httpd receiving on socket 34
conn_request

POST /feedback RTSP/1.0
CSeq: 13
DACP-ID: 7C6F7F463A6E6387
Active-Remote: 151891940
User-Agent: AirPlay/620.8.2

Handling request POST with URL /feedback
raop_handler_feedback

RTSP/1.0 200 OK 
CSeq: 13 
Server: AirTunes/220.68 

Removing connection for socket 34
Destroying connection
raop_rtp video: now = 1666680298.248710, ntp = 1666680298.347224, latency = -0.098514
Open connections: 1
raop_rtp_mirror->running is no longer true
raop_rtp_mirror exiting TCP thread
raop_ntp stopping time thread
raop_ntp exiting thread
raop_ntp stopped time thread
Removing connection for socket 36
Destroying connection

[fduncanh]

I tested (using the current gstreamer1.0-plugins-good from Raspberry Pi OS)

your problem occurs when the -v4l2 option is used, but not without it. The -v4l2 option uses the gpu for "videoconvert" as well as for h264 decoding.

When I use the patched gstreamer1.0-plugins-good (patch v4l2_1_18_4_diff3.txt from the uxplay wiki instructions) there is no problem with -v4l2.

So I guess that the patch that Raspberry Pi OS applied is not as complete as the the one in the uxplay wiki. (I had thought their most recent patch was equivalent to my v4l2_1_18_4_diff2.txt patch.).

In any case, "uxplay -bt709 -vs kmssink" (without the -v4l2) seems to work OK when the screen is rotated in Raspberry Pi OS "Lite" without extra patching, so you don't need to do the wiki patch to gstreamer1.0-plugins-good (if you do the patch, you will not need the -bt709 any more).

(Please confirm this fixes your problem, and close the issue if it does)

EDIT: this issue occurs with the "Desktop" version of Raspberry PI OS as well: don't use -v4l2 for the unpatched system.

[gregd72002]

Thanks. After recompiling gstreamer1.0-plugins-good with patch v4l2_1_18_4_diff3.txt, I can now rotate the screen. Now I still need to resolve issue with no audio. But I think it is down to my setup in here.