vaapi hardware decode fails with mp4 with particular colorspace

[RKelln]

Having more trouble with vaapi hardware decoding:

0:00:00.213950956 615566 0x7f398400b180 ERROR       vaapivideomemory gstvaapivideomemory.c:254:map_vaapi_memory: failed to make image current
0:00:00.214026820 615566 0x7f398400b180 ERROR                default video-frame.c:168:gst_video_frame_map_id: failed to map video frame plane 0

This happens with mp4 with ffprobe stream info:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'not_working.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.76.100
  Duration: 00:05:07.17, start: 0.000000, bitrate: 13467 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 13464 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)

But not mp4 with this probe info:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'works.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.16.100
  Duration: 00:05:08.61, start: 0.000000, bitrate: 12259 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 1920x384 [SAR 1:1 DAR 5:1], 12249 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)

I'm having a hard time getting ffmpeg to convert a single file to different color spaces so a bit hard to test. Also note that a hevc encoded version of the failing file works with ffprobe info:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'hevc_works.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2mp41
    encoder         : Lavf58.76.100
  Duration: 00:00:10.00, start: 0.000000, bitrate: 6737 kb/s
  Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 6731 kb/s, 30 fps, 30 tbr, 15360 tbn, 30 tbc (default)

So it seems to be related to the colorspace? Any ideas?

[brunoherbelin]

I'm sorry, but I can't solve bugs for gstreamer and for the vaapi drivers :disappointed: And yes, its not the first time people report unsupported formats that gstreamer decodebin decides to decode with vaapi decoder.

The only thing that vimix can do is to prevent gstreamer from choosing vaapi decoder to open a specific video file. The procedure for vimix 0.8 and bellow is as follows

1. disable hardware decoding in general vimix Settings. Restart vimix

2. open the session where to put the video that does not load. Open the video (it should work with software decoder)

3. In the Player options for this video, disable hardware decoding:

   

4. Save the session

5. If you re-enable hardware decoding and restart vimix, this specific video will NOT use hardware decoding (and should play

Facing this new recurrent issue (vaapi decoding didn't cause this issue before), I am wandering how to help vimix users to make this easier. Maybe a switch to disable hardware decoding only for openning one specific video? but then where to put it and how be clear for the user that this is only temporary? Or the opposite, default to software decoding and ask for enabling hardware ?

Your input is welcome on this user experience matter.

[RKelln]

Yup, I can try to report to gstreamer. I just checked and it doesn't happen in the flatpak, just in my local build. So not sure what that means. I thought the versions of gstreamer were the same... hmmm, no, off by a single patch version and missing some vaapi features. Argh.

Local build:

GStreamer 1.20.03
Plugins & features (runtime) :
...
> vaapi
 -   vaapiav1dec
 -   vaapidecodebin
 -   vaapih264dec
 -   vaapih264enc
 -   vaapih265dec
 -   vaapih265enc
 -   vaapijpegdec
 -   vaapijpegenc
 -   vaapimpeg2dec
 -   vaapipostproc
 -   vaapisink
 -   vaapivp8dec
 -   vaapivp9dec
 -   vaapivp9enc

(Missing: vaapimpeg2enc, vaapioverlay, vaapivc1dec from below)

Flatpak:

GStreamer 1.20.04
Plugins & features (runtime) :
...
> vaapi
 -   vaapiav1dec
 -   vaapidecodebin
 -   vaapih264dec
 -   vaapih264enc
 -   vaapih265dec
 -   vaapih265enc
 -   vaapijpegdec
 -   vaapijpegenc
 -   vaapimpeg2dec
 -   vaapimpeg2enc
 -   vaapioverlay
 -   vaapipostproc
 -   vaapisink
 -   vaapivc1dec
 -   vaapivp8dec
 -   vaapivp9dec
 -   vaapivp9enc

I'm on Ubuntu 22.04, with gstreamer just installed normally, so dunno why the versions and plugins would be different?

In any case, the software per file workaround seems good. UI/workflow-wise, having a per source hardware toggle I would have it on the panel that shows when you select a source. And when you load a source that fails, bring up that panel automatically? Or also ask (if hardware accel enabled) to try to reload a failed source in the warning dialog?

Thanks!

[RKelln]

I don't see the hardware decoding option in that drop down menu?

[RKelln]

Ah, the hardware decoding option is only available when hardware en/decode is enabled, but the file can't be in the Player when it can't open, so there is no way to per file disable hardware decoding.

[RKelln]

OK, took a look at this, it's weirder than we thought. It was the gst discoverer failing, so even setting the videos to software decode wouldn't work. When gst discoverer done manually it also reports an error but also the video data:

$ gst-discoverer-1.0 not_working.mp4 
Analyzing file:///.../not_working.mp4
Done discovering file:///.../not_working.mp4
An error was encountered while discovering the file
 Internal data stream error.

Properties:
  Duration: 0:05:07.167000000
  Seekable: yes
  Live: no
  container #0: Quicktime
    video #1: H.264 (High Profile)
      Stream ID: 45f96e546fad7f70a98402c310ba749e6ecc188b3f4da0b84b7118015397586b/001
      Width: 1920
      Height: 1080
      Depth: 24
      Frame rate: 30/1
      Pixel aspect ratio: 1/1
      Interlaced: false
      Bitrate: 13464348
      Max bitrate: 13464348

So just allowing the discoverer process to continue then allows these videos to play with software decode. Made a quick commit with an example that allows software decoding of these problematic files: https://github.com/RKelln/vimix/commit/26cb75f93fe40230de4f4a111827a777b6b96d8c

[brunoherbelin]

Hi ! Thanks for investigating the issue and looking for options.

1. Yes, sorry, you are right, if hw decoding is disabled globally, then it cannot be disabled locally in the player... stupid of me :) So yes, indeed, an option elsewhere is necessary. I started to look at it and will improve very soon
2. automatic detection of failure to load with hw decoding and switch to software is the plan, but not so easy...
3. excellent that you propose fix in your repo !! Thanks. I will check and we'll fix this for sure :)

[brunoherbelin]

Hopefully resolving this issue of hardware decoding for some videos;

In commit ced318637fdddf9514ecaa76ad5dd887914165ad there is a new button allowing to try again loading a file that failed:

This buttons is only appearing if the global settings enables hardware decoding, and will set the flag 'hardware decoding' to 'Disabled' for that video.

This way, if for any reason the vaapi or other hardware decoding fails, the source is possibly loaded with software decoder. This also makes it clear for the user that the acceleration is not used (i.e. not hidden by an automatic fix), thus suggesting to do something if hw decoding is wanted (e.g. change encoder, fix drivers).

[RKelln]

Yeah, good fix, that makes it clear and easy. When I tried it with the colorspace failure files it didn't detect the failure, the thumbnails were transparent, but I was able to use the Player Video menu to disable hardware and get it visible.