Hardware decoding using gstreamer vaapi no longer working

[sdreisei]

On my up-to-date arch linux system running on a Gen1 Thinkpad 14s with AMD Ryzen 7 and Radeon graphics, uxplay was working great up to (and including) v. 1.48 via the command uxplay -p -vs vaapisking. In 1.49 and 1.50, the gstreamer window that opens for mirroring is black (for a split second, I can see my mirrored iPad screen). For hardware video decoding, I can only use the vaapisink video sink, as the others available on my system result in an X11 BadWindow error. Software decoding with the -avdec switch still works, so this is a minor issue, but I still wanted to point it out.

Here's the output I receive, which does not show any differences between 1.48 and 1.50:

thinkpad ~/build/UxPlay-master> ./uxplay -p -vs vaapisink
using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address xx:xx:xx:xx:xx:xx
Initialized server socket(s)
Accepted IPv4 client on socket 31
Local: 192.168.134.41
Remote: 192.168.134.89
Open connections: 1
Client identified as User-Agent: AirPlay/605.1
Accepted IPv4 client on socket 33
Local: 192.168.134.41
Remote: 192.168.134.89
Open connections: 2
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline
raop_rtp_mirror->running is no longer true
Connection closed for socket 31
Destroying connection
Open connections: 1
Connection closed for socket 33
Destroying connection
Open connections: 0
^CStopping...

The first part of the vaapisink information on my system is

thinkpad ~/build/UxPlay-master> gst-inspect-1.0 vaapisink
Factory Details:
  Rank                     secondary (128)
  Long-name                VA-API sink
  Klass                    Sink/Video
  Description              A VA-API based videosink
  Author                   Gwenole Beauchesne <gwenole.beauchesne@intel.com>

Plugin Details:
  Name                     vaapi
  Description              VA-API based elements
  Filename                 /usr/lib/gstreamer-1.0/libgstvaapi.so
  Version                  1.20.1
  License                  LGPL
  Source module            gstreamer-vaapi
  Source release date      2022-03-14
  Binary package           gstreamer-vaapi
  Origin URL               https://www.archlinux.org/

My gstreamer version is

thinkpad ~/build/UxPlay-master> gst-launch-1.0 --gst-version
GStreamer Core Library version 1.20.1

What other information can I provide you with? Many thanks for maintaining uxplay!

[fduncanh]

Thanks for reporting this. I see you are using latest gstreamer 1.20.1

I cant easily test vaapi without removing an nvdia card.

Please do a fresh build with 1.48 to confirm it still works for you (in case anything changed on your system since you first build uxplay-1.48. You can get it in the "releases" panel on github.

Also start uxplay -d : immediately after starting it will show the gstreamer video pipeline that will be used - in each case. You dont need to connect after that, just verify the pipeline that will be used and post it here.

I will then give you further instructions depending on what I see,

[fduncanh]

You said uxplay 1.48 was working for you. Get 1.48e of March 12 2022 from the releases, as a zip file. build and test that this is working https://github.com/FDH2/UxPlay/releases/tag/v1.48e

It this works but release 1.49 https://github.com/FDH2/UxPlay/releases/tag/v1.49 doesnt it will be easy to fix.

[fduncanh]

In UxPlay/renderers/video_renderer_gstreamer.c, (new code) near line 145, look for

    renderer->appsrc = gst_bin_get_by_name (GST_BIN (renderer->pipeline), "video_source");
    assert(renderer->appsrc);
    caps = gst_caps_from_string(h264_caps);
    g_object_set(renderer->appsrc, "caps", caps, NULL);
    gst_caps_unref(caps);

comment out the line

  //g_object_set(renderer->appsrc, "caps", caps, NULL);

and rebuild.

Does commenting out this (recently added) line fix your vaapisink issue?

[fduncanh]

what linux distribution are you running?

[fduncanh]

Okay I investigated, and did not find any problem with vaapisink. with latest github uxplay or 1.49 or 1.48-e

I removed a nvidia graphics card and tested with vaapi.

This is intel, not amd. It works just fine. In fact the image is very good. The new -fs (fullscreen) option for vaapi is working as expected.

I used a new installation (to avoid messing up an nvidia installation) of mint20.3 cinnamon which has a rather old gstreamer-1.16.2.

Perhaps there is some issue with gstreamer-1.20? Do you run something like Manjaro trhat provides gstreamer-1.20.1 or did you build it yourself?

Please check that a new (re)build of uxplay -1.48 still works on your system, and provide more details. (if not, something was messed on your system up after the older build)

If a new build of older code works and a new build of later coder doesn't, you can find easily find which change broke things for you by building and testing each change in github to find when the problem appears.

click on "commits" on the github uxplay home page, for each commit, the < > button give access to the code as of that commit, and you can download it as a zip file. (note most recent commits are just edits to the README, bother only with commits that make a code change)

Please find when your error appears by downloading a version as each code change is made, working forwards from a working version in the past, fresh build on your current system.

[fduncanh]

Further tests: I have now tested latest uxplay -vs vaapisink with Manjaro Linux which uses gstreamer-1.20.1,

uname -a                                                                                                                                       ✔ 
Linux elite 5.15.32-1-MANJARO #1 SMP PREEMPT Mon Mar 28 09:16:36 UTC 2022 x86_64 GNU/Linux

Works perfectly. (intel graphics). So the source of your issue is not gstreamer-1.20.1

Definitely please confirm that a new build of uxplay 1.48 or earlier works but 1.49 doesnt. , on your system.

I see you are also using aarch or an aarch derivative, as I now am.

AHA! are you building from github, or using AUR? maybe the AUR build is bad somehow? Please test with a github build, not AUR. EDIT: NO, both AUR packages ("1.49" and "github") work perfectly with vaapsink on my Manjaro test system

gst-inspect-1.0 vaapisink                                                                                                                      ✔ 
Factory Details:
  Rank                     secondary (128)
  Long-name                VA-API sink
  Klass                    Sink/Video
  Description              A VA-API based videosink
  Author                   Gwenole Beauchesne <gwenole.beauchesne@intel.com>

Plugin Details:
  Name                     vaapi
  Description              VA-API based elements
  Filename                 /usr/lib/gstreamer-1.0/libgstvaapi.so
  Version                  1.20.1
  License                  LGPL
  Source module            gstreamer-vaapi
  Source release date      2022-03-14
  Binary package           gstreamer-vaapi
  Origin URL               https://www.archlinux.org/

[sdreisei]

Thank you for all the effort you put into this. Let me try to respond to all your comments here. When I wrote "1.48", I actually meant 1.48e, as this is the version available under the heading "UxPlay v1.48" on github. The start of the output of uxplay -d is

supported audio format 1: AAC-ELD 44100/2
supported audio format 2: ALAC 44100/16/2
supported audio format 3: AAC-LC 44100/2
supported audio format 4: PCM 44100/16/2 S16LE
GStreamer video pipeline will be:
"appsrc name=video_source stream-type=0 format=GST_FORMAT_TIME is-live=true ! queue ! h264parse ! decodebin ! videoconvert ! autovideosink name=video_sink sync=false"
Initialized GStreamer video renderer
using system MAC address xx:xx:xx:xx:xx:xx
Initialized server socket(s)
^CStopping...

with autovideosink replaced by vaapisink if I start uxplay -d -vs vaapisink. This does not change for versions 1.48e, 1.49 or 1.50.

Commenting out the line in UxPlay/renderers/video_renderer_gstreamer.c that you indicated did not change anything for me. I can confirm that a new build of 1.48e still works with vaapi, but a new build from 1.49 on does not. I am on an arch linux system with current kernel

Linux thinkpad 5.17.3-arch1-1 #1 SMP PREEMPT Thu, 14 Apr 2022 01:18:36 +0000 x86_64 GNU/Linux

and gstreamer provided by arch. In fact, UxPlay is the only software I built from sources. I am building from both AUR and the github sources, all the outputs reported here are from the github sources. I used them after I suspected something wrong with AUR, but 1.50 (github), 1.49 (github) and 1.50 (AUR) all give the same black screen.

Since it works on your Intel graphics, but not on my AMD Radeon, there might be an issue with the Radeon chips. I have a version of the software that works for me, and if nobody else has the same issues, it might not be worth your time to go down some rabbit hole (although I certainly appreciate your time on this).

[fduncanh]

@sdreisei If 1.48e works for vaapi/AMD and 1.49 doesn't, there is a regression I would really want to fix. Please don't give up on this, I need your help. (This should be easy, and is quite different from trying to get something that never worked to work)

If vaapi really worked for 1.48e and not for 1.49, the only change that seems relevant is

--- UxPlay-1.48e/renderers/video_renderer_gstreamer.c   2022-03-21 11:34:45.000000000 -0400
+++ UxPlay-1.49/renderers/video_renderer_gstreamer.c    2022-04-01 15:23:05.000000000 -0400
@@ -96,6 +96,7 @@
 static logger_t *logger = NULL;
 static unsigned short width, height, width_source, height_source;  /* not currently used */
 static bool first_packet = false;
+static const char h264_caps[]="video/x-h264,stream-format=(string)byte-stream,alignment=(string)au";

 void video_renderer_size(float *f_width_source, float *f_height_source, float *f_width, float *f_height) {
     width_source = (unsigned short) *f_width_source;
@@ -108,6 +109,7 @@
 void  video_renderer_init(logger_t *render_logger, const char *server_name, videoflip_t videoflip[2], const char *parser,
                           const char *decoder, const char *converter, const char *videosink) {
     GError *error = NULL;
+    GstCaps *caps = NULL;
     logger = render_logger;

     /* this call to g_set_application_name makes server_name appear in the  X11 display window title bar, */
@@ -139,6 +141,10 @@

     renderer->appsrc = gst_bin_get_by_name (GST_BIN (renderer->pipeline), "video_source");
     assert(renderer->appsrc);
+    caps = gst_caps_from_string(h264_caps);
+    g_object_set(renderer->appsrc, "caps", caps, NULL);
+    gst_caps_unref(caps);
+
     renderer->sink = gst_bin_get_by_name (GST_BIN (renderer->pipeline), "video_sink");
     assert(renderer->sink);

This is not in any way an important change, and is removable.

(1) Please edit 1.49 to remove or comment out ("//...") ALL the "+" lines, and rebuild. Does it now work?

(2) for the "working" 1.48, please run with -vs "vaapisink fullscreen=true" to check that you get fullscreen (this is just a check that vaapi really is being used)

(3) for the nonworking 1.49, please do "export GST_DEBUG=2" before running and post the GStreamer debug output here.

(4) repeat (3) but with GST_DEBUG=4: there will be a LOT of output. Post it as a file that you drag and drop here

[sdreisei]

I'm glad to be of assistance!

(1) Please edit 1.49 to remove or comment out ("//...") ALL the "+" lines, and rebuild. Does it now work?

No, unfortunately not :(

(2) for the "working" 1.48, please run with -vs "vaapisink fullscreen=true" to check that you get fullscreen (this is just a check that vaapi really is being used)

Yes, I get fullscreen with this option.

(3) for the nonworking 1.49, please do "export GST_DEBUG=2" before running and post the GStreamer debug output here.

0:00:00.081071259 786848 0x563d09cce100 WARN              vaapiblend gstvaapiblend.c:184:gst_vaapi_blend_initialize:<vaapiblend0> VPP does not support global alpha blending
using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address xx:xx:xx:xx:xx:xx
Initialized server socket(s)
Accepted IPv4 client on socket 31
Local: 10.20.99.92
Remote: 10.20.99.27
Open connections: 1
Client identified as User-Agent: AirPlay/610.19.1
Accepted IPv4 client on socket 33
Local: 10.20.99.92
Remote: 10.20.99.27
Open connections: 2
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline
0:00:10.063241961 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2393:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support '1:3:5:1' input colorimetry. vpp may fail or produce unexpected results.
0:00:10.063268152 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2403:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' output colorimetry. vpp may fail or produce unexpected results.
0:00:10.078211302 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2393:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support '1:3:5:1' input colorimetry. vpp may fail or produce unexpected results.
0:00:10.078248597 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2403:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' output colorimetry. vpp may fail or produce unexpected results.
raop_rtp_mirror tcp socket is closed, connection ended
Connection closed for socket 31
Destroying connection
Open connections: 1
Connection closed for socket 33
Destroying connection
Open connections: 0

(4) repeat (3) but with GST_DEBUG=4: there will be a LOT of output. Post it as a file that you drag and drop here

See the attached file. Hope this helps! uxplay_GST_DEBUG_4_output.txt

[fduncanh]

Thanks ! I'll take a look.

But I also just realized there was an available test amd machine (pretty old, c. 2009 Phenom X4 II 830, with "Radeon HD 6450/7450/8450 R5 230 OEM" Graphics. running kernel 5.3.18 Gstreamer 1.16.3.

vaapi works fine with both uxplay 1.48-e and 1.49 (and current github) on this old beast!

[fduncanh]

However, I think I see the problem you have! (and know the fix)..... The same thing came up in support for the GPU on Rasperry Pi with Video4linux2.

driver does not support '1:3:5:1' input colorimetry. vpp may fail or produce unexpected results.
0:00:10.063268152 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2403:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' output colorimetry. vpp may fail or produce unexpected results.
0:00:10.078211302 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2393:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support '1:3:5:1' input colorimetry. vpp may fail or produce unexpected results.
0:00:10.078248597 786848 0x563d09b66f60 WARN             vaapifilter gstvaapifilter.c:2403:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' output colorimetry. vpp may fail or produce unexpected results.

I think this may fix it....

 uxplay - vp "h264parse ! capssetter caps=\"video/x-h264, colorimetry=bt709\"" -vs vaapisink

Its probably an issue with the combination GStreamer-1.20.1 and AMD Radeon GPU which I dont have available right now.

[fduncanh]

If that fix works, try adding it to the line

--- UxPlay-1.48e/renderers/video_renderer_gstreamer.c 2022-03-21 11:34:45.000000000 -0400 +++ UxPlay-1.49/renderers/video_renderer_gstreamer.c 2022-04-01 15:23:05.000000000 -0400 @@ -96,6 +96,7 @@ static logger_t logger = NULL; static unsigned short width, height, width_source, height_source; / not currently used */ static bool first_packet = false; +static const char h264_caps[]="video/x-h264,stream-format=(string)byte-stream,alignment=(string)au";

making it static const char h264_caps[]="video/x-h264,colorimetry=bt709,stream-format=(string)byte-stream,alignment=(string)au";

[sdreisei]

I think this may fix it....

uxplay - vp "h264parse ! capssetter caps=\"video/x-h264, colorimetry=bt709\"" -vs vaapisink

Its probably an issue with the combination GStreamer-1.20.1 and AMD Radeon GPU which I dont have available right now.

Unfortunately not... Here's the output, again with GST_DEBUG=2:

./uxplay -p -vp "h264parse ! capssetter caps=\"video/x-h264, colorimetry=bt709\"" -vs vaapisink
0:00:00.079732480 814398 0x55b4a3ec4c60 WARN              vaapiblend gstvaapiblend.c:184:gst_vaapi_blend_initialize:<vaapiblend0> VPP does not support global alpha blending
using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address xx:xx:xx:xx:xx:xx
Initialized server socket(s)
Accepted IPv4 client on socket 31
Local: 10.20.99.92
Remote: 10.20.99.27
Open connections: 1
Client identified as User-Agent: AirPlay/610.19.1
Accepted IPv4 client on socket 33
Local: 10.20.99.92
Remote: 10.20.99.27
Open connections: 2
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline
0:00:11.089574629 814398 0x55b4a3e20b60 WARN             vaapifilter gstvaapifilter.c:2393:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' input colorimetry. vpp may fail or produce unexpected results.
0:00:11.089600401 814398 0x55b4a3e20b60 WARN             vaapifilter gstvaapifilter.c:2403:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' output colorimetry. vpp may fail or produce unexpected results.
0:00:11.106226455 814398 0x55b4a3e20b60 WARN             vaapifilter gstvaapifilter.c:2393:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' input colorimetry. vpp may fail or produce unexpected results.
0:00:11.106255579 814398 0x55b4a3e20b60 WARN             vaapifilter gstvaapifilter.c:2403:gst_vaapi_filter_set_colorimetry_unlocked:<vaapifilter1> driver does not support 'bt709' output colorimetry. vpp may fail or produce unexpected results.
raop_rtp_mirror tcp socket is closed, connection ended
Connection closed for socket 31
Destroying connection
Open connections: 1
Connection closed for socket 33
Destroying connection
Open connections: 0

[fduncanh]

Just checking, this is pure 1.49 or github with no changes?

Do you also get those messages with GST_DEBUG=2 and 1.48?

[sdreisei]

Yep, it's the exact same output on vanilla 1.48e, 1.49 and 1.50, all from github. It works on 1.48e, but does not on the other two (so even on a working version I get the colorimetry warning).

Strange thing, it did work exactly once on both 1.49 and 1.50, but when I tried it again, it failed, and kept failing...

[fduncanh]

The colorimetry warning is probably not the issue then.

The most useful thing you could do would be to look for the point in github between 1.48-3 (March 21) and 1.49 (April 1) where things stop working. Go to the links and then click on "browse files" to access the code at that date to download a zip file March 21 1.48-e https://github.com/FDH2/UxPlay/commit/23c2d263f2d8bd360069cbef787c73c5fb0fd67c

march 24 https://github.com/FDH2/UxPlay/commit/f293a22b920f2e38b3d1b213aca8f6087661bdb7

march 24 https://github.com/FDH2/UxPlay/commit/7c8636aa4f7d085acb49c6812bf30249a00adf58

march 28 https://github.com/FDH2/UxPlay/commit/aec4a073466d1c1545ee480dfba725fae34595f8

April 1 1.49 https://github.com/FDH2/UxPlay/commit/db802cc2a89918b8640aceeebdc34d3cd9cd1257

[azuwis]

@sdreisei You can use https://git-scm.com/docs/git-bisect to find out the commit cause the problem.

There are 22 commits between v1.48e to v1.49, it should take less than 5 runs to find out.

[fduncanh]

And most of the commits are documentation edits or whitespace cleanups

[sdreisei]

Sounds like a plan... I'll see what I can do with the links posted above, then brush up on my git commands---it's been quite a while. Probably will only be able to start on this tomorrow, it's getting late here (although I am curious).

[azuwis]

Step 1:

git bisect start
git bisect bad v1.49
git bisect good v1.48e

Step 2:

cmake
make
# test, based on the result, run
git bisect good
# or
git bisect bad

Repeat step 2.

[sdreisei]

Wow, that worked like a charm. I discovered (and verified with explicit git checkout afterwards) that 17a2b2564843c6640f98f116275e9fec4ecbca04 is the last good commit, and 695da77f56b905c025b1fc40c3c71e4bee638c17 is the first bad.

Just in case you're interested, here's the full log:

git bisect start
# bad: [db802cc2a89918b8640aceeebdc34d3cd9cd1257] fix typo that broke code
git bisect bad db802cc2a89918b8640aceeebdc34d3cd9cd1257
# good: [23c2d263f2d8bd360069cbef787c73c5fb0fd67c] edit README
git bisect good 23c2d263f2d8bd360069cbef787c73c5fb0fd67c
# good: [7fb0dec4b13223535059fa3c44d43912805ca73e] minor README edits
git bisect good 7fb0dec4b13223535059fa3c44d43912805ca73e
# bad: [f98baedd48a5136d90f9ea4020025542d9a8a7c5] whitespace cleanup
git bisect bad f98baedd48a5136d90f9ea4020025542d9a8a7c5
# good: [17a2b2564843c6640f98f116275e9fec4ecbca04] uxplay.cpp fix missing variable in err  message
git bisect good 17a2b2564843c6640f98f116275e9fec4ecbca04
# bad: [695da77f56b905c025b1fc40c3c71e4bee638c17] fixed M1 video problem
git bisect bad 695da77f56b905c025b1fc40c3c71e4bee638c17

[fduncanh]

Thanks!

Still difficult to understand why this breaks things for you.

What kind of video were you mirroring? was the client iOS or a Mac (if mac, was it intel or M1, etc)

[sdreisei]

The client is a 2020 iPad Air with updated iPadOS (currently 15.4.1). I'm using the iPad as a whiteboard to write on while lecturing, and screen-mirroring its contents to my arch linux thinkpad. So it's not movies or anything with high-bandwidth changes between frames.

[fduncanh]

but everything works when not using vaapi, right ?

[fduncanh]

could I get some uxplay -d debug output to see the beginning of the video streaming output

[sdreisei]

uxplay-d_17a2b2564843c6640f98f116275e9fec4ecbca04.txt uxplay-d_695da77f56b905c025b1fc40c3c71e4bee638c17.txt On the last-known-good commit 17a2b2564843c6640f98f116275e9fec4ecbca04, this is

./uxplay -p -d -vs vaapisink
supported audio format 1: AAC-ELD 44100/2
supported audio format 2: ALAC 44100/16/2
supported audio format 3: AAC-LC 44100/2
supported audio format 4: PCM 44100/16/2 S16LE
GStreamer video pipeline will be:
"appsrc name=video_source stream-type=0 format=GST_FORMAT_TIME is-live=true ! queue ! h264parse ! decodebin ! videoconvert ! vaapisink name=video_sink sync=false"
Initialized GStreamer video renderer
using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address 8c:8c:aa:1d:98:82
Initialized server socket(s)
Accepted IPv4 client on socket 31
Local: 10.20.99.92
Remote: 10.20.99.30
Open connections: 1
httpd receiving on socket 31
conn_request

GET /info RTSP/1.0
X-Apple-ProtocolVersion: 1
Content-Length: 70
Content-Type: application/x-apple-binary-plist
CSeq: 0
DACP-ID: 650317F24B3E9E3E
Active-Remote: 1624187470
User-Agent: AirPlay/610.19.1

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>qualifier</key>
    <array>
        <string>txtAirPlay</string>
    </array>
</dict>
</plist>

Handling request GET with URL /info

RTSP/1.0 200 OK 
CSeq: 0 
Server: AirTunes/220.68 
Content-Type: application/x-apple-binary-plist 
Content-Length: 1063 

on the first bad commit 695da77f56b905c025b1fc40c3c71e4bee638c17, its

./uxplay -p -d -vs vaapisink
supported audio format 1: AAC-ELD 44100/2
supported audio format 2: ALAC 44100/16/2
supported audio format 3: AAC-LC 44100/2
supported audio format 4: PCM 44100/16/2 S16LE
GStreamer video pipeline will be:
"appsrc name=video_source stream-type=0 format=GST_FORMAT_TIME is-live=true ! queue ! h264parse ! decodebin ! videoconvert ! vaapisink name=video_sink sync=false"
Initialized GStreamer video renderer
using network ports UDP 7011 6001 6000 TCP 7100 7000 7001
using system MAC address 8c:8c:aa:1d:98:82
Initialized server socket(s)
Accepted IPv4 client on socket 31
Local: 10.20.99.92
Remote: 10.20.99.30
Open connections: 1
httpd receiving on socket 31
conn_request

GET /info RTSP/1.0
X-Apple-ProtocolVersion: 1
Content-Length: 70
Content-Type: application/x-apple-binary-plist
CSeq: 0
DACP-ID: CC9AE79D45303A35
Active-Remote: 3884623255
User-Agent: AirPlay/610.19.1

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>qualifier</key>
    <array>
        <string>txtAirPlay</string>
    </array>
</dict>
</plist>

Handling request GET with URL /info

RTSP/1.0 200 OK 
CSeq: 0 
Server: AirTunes/220.68 
Content-Type: application/x-apple-binary-plist 
Content-Length: 1063 

[sdreisei]

On the "everything works without vaapi" question: On 1.50, uxplay does not work, uxplay -p -avdec however does. In yesterday's lecture, some versions that I tried with -avdec crashed with segfaults after some minutes (did not write down which these were, sorry).

[fduncanh]

Its a mystery so far!

The very latest github now has a fix for the colorimetry =1:3:5:1 issue (previously it was only used for the RPi) . Id like to see an output from

export GST_DEBUG=4
uxplay -p -d -vs vaapisink

to see the failure details when the colorimetry fix is applied.

[sdreisei]

On the latest commit 052c1dd36d49b58ca0155bc74382690182590511, here's what I get. uxplay.txt

[azuwis]

Maybe we should use vah264dec from gst-plugins-bad, according to https://github.com/Rafostar/clapper/wiki/Hardware-acceleration

AMD/Intel

GStreamer now has two different plugins that provide hardware acceleration vaapi (provided by gstreamer-vaapi) and newer va (part of gst-plugins-bad). Starting with GStreamer 1.19.1, Clapper gradually enables and uses new va decoders by default. Otherwise older vaapi plugins are used. You can control which one to use in Clapper preferences "Plugin ranking" by changing their default rank to be higher then alternative (e.g. apply rank 300 to vah264dec and so on).

On some older GPUs you might need to export GST_VAAPI_ALL_DRIVERS=1 environment variable.

I've try gst-launch-1.0 -v filesrc location=/foo.mp4 ! parsebin ! vah264dec ! autovideosink, it works on wayland with hardware acceleration (tell by intel_gpu_top).

[azuwis]

I can confirm that hardware decoding works with uxplay -p -vd vah264dec.

[sdreisei]

Do you mean with the gst-plugins-bad modifications? I haven't tried these yet, but the latest commit fails for uxplay -p -vd vah264dec with

Open connections: 2
raop_rtp_mirror starting mirroring
Begin streaming to GStreamer video pipeline
GStreamer error: Internal data stream error.
Removing connection for socket 27
Destroying connection

Edit: I think I understand now, the vah264dec option is the gst-plugins-bad modification.

[azuwis]

Try set GST_VAAPI_ALL_DRIVERS=1 environment variable

[sdreisei]

Too bad, still does not work, even when I set GST_VAAPI_ALL_DRIVERS=1, with the exact same error message. Strange enough, when I tried gst-launch-1.0 -v filesrc location=/foo.mp4 ! parsebin ! vah264dec ! autovideosink after setting the environment variable above, I get WARNING: erroneous pipeline: no element "vah264dec", although uxplay does not complain about the -vd vah264dec switch.

[fduncanh]

I'll sort this out in a day or two maybe

[sdreisei]

Thanks alot for all your efforts, I really appreciate it!

[azuwis]

Too bad, still does not work, even when I set GST_VAAPI_ALL_DRIVERS=1, with the exact same error message. Strange enough, when I tried gst-launch-1.0 -v filesrc location=/foo.mp4 ! parsebin ! vah264dec ! autovideosink after setting the environment variable above, I get WARNING: erroneous pipeline: no element "vah264dec", although uxplay does not complain about the -vd vah264dec switch.

Have you installed gst-plugins-bad? https://archlinux.org/packages/extra/x86_64/gst-plugins-bad/

[sdreisei]

Yes I have, along with gstreamer-vaapi, although I seem to recall that the --vaapisink switch used to work even without explicitly installing that package.

[azuwis]

Archlinux split libgstva.so into gst-plugin-va https://archlinux.org/packages/extra/x86_64/gst-plugin-va/

Check if you have installed it.

[sdreisei]

Nope, I hadn't had that one. I installed it, and now it get the same black screen with -vd vah264dec that I get with -vs vaapi (so no longer an error message). With this latest install, gst-launch-1.0 -v filesrc location=/foo.mp4 ! parsebin ! vah264dec ! autovideosink works, so that's progress :)

[fduncanh]

The obvious clue is that it stopped working at a particular commit.... I will try to break that commit up into a sequence of smaller patches to see if that isolates it

But right now I'm still studying the GST_DEBUG log

[fduncanh]

@sdreisei

Hi again, can you apply the following three patches to the last working version 17a2b2564843c6640f98f116275e9fec4ecbca04

diff1.txt

diff2.txt

diff3.txt

(patch p1 < (path_to/diff1.txt in the UxPlay-17a2..... directory) etc

They apply in order. check whether uxplay is still working with -vs vaapisink after each patch

Thank you so much!

[sdreisei]

The first two went okay, it broke on the third patch. Interesting collaborative detective work :)

[fduncanh]

Yes ! that's where I thought the issue might be: some thing that needs to be used later was stored, but I moved it to a place that it seems doesnt preserve it long enough. my mistake. It could be a race condition so it fails on some systems but not others.

[fduncanh]

@sdreisei

Before I declare victory on this, can you test with a replacement for diff3.txt: Do the sequence diff1.txt diff2.txt and then (instead)

diff3a.txt

The diff3 I sent earlier had an error that looks to me like it should have broken things anyway, (but doesnt when I tested it).

If the new version diff3a.txt doesnt break it for you, the error is still further along the chain of changes. (so I hope diff3a breaks things!)

[sdreisei]

Sorry, but it still works after applying the diff3a.txt patch...

[fduncanh]

Well that diff3a did not break it for me either.

I've updated current github , please test to see if diff1 +diff2 were in fact the fix.

[sdreisei]

Congrats, yes, the latest commit 5a9ffc2f8847dae25ce5cf2f5d21b99392777745 works for me!

[fduncanh]

Solved! Happy lecturing! report any other issues!