search

brunoherbelin / vimix

Live Video Mixer
GNU General Public License v3.0
259 stars 25 forks source link

Hardware acceleration issue on Ubuntu with Intel vaapi #63

Closed RKelln closed 1 year ago

RKelln commented 1 year ago

Having two different but potentially linked issues on a Ubuntu 22.04 machine with Intel CPU and graphics.

1) The snap (0.7.1) has black only output window (with both hardware accelerated and not). Says Gstreamer 1.14.05 in About. 2) Compiled latest (0.7.3), which has working output window but only loads videos (only tested mp4 so far) when no hardware acceleration used. Says GStreamer 1.20.03 in About.

It is quite likely that none of this has to do with vimix?

So for the compiled build: everything went fine. Videos refuse to load if hardware acceleration is on, log says:

0001  Refusing stream requests to vimix@framework.0. No streaming ongoing.
0002  Metronome Ableton Link Enabled
0003  Metronome Ableton Link start & stop sync Enabled
0004  Found the following GPU decoding plugin(s):
0005  1. vaapidecodebin
0006  OSC: Loaded 1 translation.
0007  OSC: Listening to UDP messages sent to 127.0.0.1:7000
0008  Font size 32
0009  Session 674083993530477882 Opened '.... .mix' (1 sources)
0010   Warning - 'file:///.../video.mp4': Internal data stream error.
0011   Session '....mix' loaded with 1 source.
0012  MediaPlayer 674084088962751893 Opened 'video.mp4' (H.264 (High Profile) 1920 x 1080)
0013  MediaPlayer 674084088962751893 Timeline [0 307179000000] 9215 frames, 0 gaps
0014  MediaPlayer 674084088962751893 Uses vaapidecodebin decoding and OpenGL PBO texturing.
0015  Source 'video' linked to Media 674084088962751893.

I don't know much about vaapi or gstreamer but took a look at https://lifestyletransfer.com/how-to-install-gstreamer-vaapi-plugins-on-ubuntu/ and tried some of the things there. Everything seems to be installed ok.

Log from snap version (same video loading fine, but no output):

0001  Refusing stream requests to vimix@framework.0. No streaming ongoing.
0002  Metronome Ableton Link Enabled
0003  Metronome Ableton Link start & stop sync Enabled
0004  OSC: Loaded 1 translation.
0005  OSC: Listening to UDP messages sent to 127.0.0.1:7000
0006  Font size 32
0007  New session ready.
0008   Session '' loaded with no source.
0009  MediaPlayer 674086998770961985 Opened 'video.mp4' (H.264 (High Profile), ISO MP4/M4A 1920 x 1080)
0010  MediaPlayer 674086998770961985 Timeline [0 307179000000] 9215 frames, 0 gaps
0011  MediaPlayer 674086998770961985 Uses software decoding and OpenGL PBO texturing.
0012  Source 'video' linked to Media 674086998770961985.
0013   Added Video File source 'video' (video.mp4 H.264, 1920 x 1080, 30fps)

Here is an example videotestsrc that also fails:

$ gst-launch-1.0 videotestsrc ! vaapisink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'vaapisink0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0", gst.vaapi.Display.GObject=(GstObject)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0";
ERROR: from element /GstPipeline:pipeline0/GstVaapiSink:vaapisink0: Internal error: could not render surface
Additional debug info:
../gst/vaapi/gstvaapisink.c(1547): gst_vaapisink_show_frame_unlocked (): /GstPipeline:pipeline0/GstVaapiSink:vaapisink0
ERROR: pipeline doesn't want to preroll.
ERROR: from element /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0: Internal data stream error.
Setting pipeline to NULL ...
Additional debug info:
../libs/gst/base/gstbasesrc.c(3127): gst_base_src_loop (): /GstPipeline:pipeline0/GstVideoTestSrc:videotestsrc0:
streaming stopped, reason error (-5)
ERROR: pipeline doesn't want to preroll.
Freeing pipeline ...

Intel/vaapi info:

$ vainfo 
libva info: VA-API version 1.14.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_14
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.14 (libva 2.12.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 22.3.1 ()
brunoherbelin commented 1 year ago

Sorry for late response! Thanks for reporting this issue.

First, the snap doesn't support hardware acceleration, as you could see in logs: MediaPlayer Uses software decoding. The switch to enable / disable is not doing anything in this case: I took note that it should just be disabled to avoid confusion.

Second, it seems you have hardware decoding available with the compiled version (0.7.3) ; MediaPlayer Uses vaapidecodebin decoding

However indeed, logs are surprising as you have both success messages MediaPlayer 674084088962751893 Opened 'video.mp4' and error messages Warning - 'file:///.../video.mp4': Internal data stream error.

It could indeed thus be a problem internal to gstreamer. The error message from vaapisink is indeed indicating such unavailability of vaapi ...

A decoding pipeline to test could look like this (I can't try here without vaapi hardware, but you see the logic) :

gst-launch-1.0 filesrc location=/Users/path/Video.mp4 \
  ! qtdemux \
  ! vaapidecodebin ! videoconvert ! videoscale ! autovideosink

if again the Internal data stream error occurs, I am afraid its a question of gstreamer and drivers...

brunoherbelin commented 1 year ago

aha, I forgot to say : i switched to packaging with Flatpak to overcomes this limitation of snap.

You can try with the published package or to create a custom flatpak

RKelln commented 1 year ago 
gst-launch-1.0 filesrc location=~/Videos/test.mp4 \
  ! qtdemux \
  ! vaapidecodebin ! videoconvert ! videoscale ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Got context from element 'vaapipostproc0': gst.gl.GLDisplay=context, gst.gl.GLDisplay=(GstGLDisplay)"\(GstGLDisplayX11\)\ gldisplayx11-0";
Got context from element 'vaapipostproc0': gst.vaapi.Display=context, gst.vaapi.Display=(GstVaapiDisplay)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0", gst.vaapi.Display.GObject=(GstObject)"\(GstVaapiDisplayGLX\)\ vaapidisplayglx0";
Redistribute latency...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
^Chandling interrupt. (9.9 %)
Interrupt: Stopping pipeline ...
Execution ended after 0:00:27.360159317
Setting pipeline to NULL ...
Freeing pipeline ...

Video plays but is entirely black. Hrm.

Flatpak loads and plays videos, both with and without hardware acceleration, although it looks like hw accel isn't actually working? Here is the log from the flatpack with hw accel on:

0001  Refusing stream requests to vimix@framework.0. No streaming ongoing.
0002  Metronome Ableton Link Enabled
0003  Metronome Ableton Link start & stop sync Enabled
0004  Found Gstreamer plugins in /app/lib/gstreamer-1.0/
0005  No GPU decoding plugin found.
0006  OSC: Loaded 1 translation.
0007  OSC: Listening to UDP messages sent to 127.0.0.1:7000
0008  Font size 32
0009  Session 677007046586700749 Opened '/.../test2.mix' (1 sources)
0010   Session '/.../test2.mix' loaded with 1 source.
0011  MediaPlayer 677007194866086274 Opened 'video.mp4' (H.264 (High Profile) 1920 x 1080)
0012  MediaPlayer 677007194866086274 Timeline [0 239979000000] 7192 frames, 0 gaps
0013  MediaPlayer 677007194866086274 Uses software decoding and OpenGL PBO texturing.
0014  Source 'video' linked to Media 677007194866086274.

Maybe flatpak doesn't have the vaapi gstreamer plugins?

Also fps with 1 video playing performance is 42 fps and 35fps with 2 videos playing. In the compiled vimix with no accel I got 29 fps with same 2 videos. ¯\_(ツ)_/¯

Sidenote: get a repeating message on the cli when running the flatpak and opening a file:

(vimix:2): GVFS-WARNING **: 15:07:26.173: The peer-to-peer connection failed: Error when getting information for file “/run/user/1000/gvfsd”: No such file or directory. Falling back to the session bus. Your application is probably missing --filesystem=xdg-run/gvfsd privileges.
RKelln commented 1 year ago

On cli I also get this error in the compiled version:

0:00:00.449680510 11717 0x7f6268007860 ERROR       vaapivideomemory gstvaapivideomemory.c:254:map_vaapi_memory: failed to make image current
0:00:00.449792479 11717 0x7f6268007860 ERROR                default video-frame.c:168:gst_video_frame_map_id: failed to map video frame plane 0

So I'm pretty sure this is some weird gsteamer vaapi issue, and I took a look through gstreamer issues and there are some vaapi related ones there but nothing that seems exactly the same. I can live with software rendering for now!

brunoherbelin commented 1 year ago

The installation of gstreamer and its modules is always delicate indeed...

Thanks to your input, I noticed now that the Flatpack rules for compiling gstreamer might forget vaapi: I will investigate.

Now, regarding performance and use of vaapi, its all on gstreamer (vimix launches and controls pipelines, but is limited in what to ask to gstreamer). An honest but sad conclusion of all your tests is that your system and gst do not support vaapi correctly...

PS: I use a machine with nvidia GPU and must admit that on this the drivers are very well optimized.

brunoherbelin commented 1 year ago

Since version 0.8.0, the Flatpak compilation includes the vaapi module of gstreamer.

Please try with the flatpak of version 0.8.0 and let me know if this solved the decoding problem.

RKelln commented 1 year ago

The flatpack is working with hardware decoding turned on (and off)! Thanks! Fps is mostly 60 too. No errors on command line, although I got a bunch on shutdown after switch from software to hardware decode (only the first time, couldn't replicate).

Log:

0001  Refusing stream requests to vimix@framework.0. No streaming ongoing.
0002  Metronome Ableton Link Enabled
0003  Metronome Ableton Link start & stop sync Enabled
0004  Found Gstreamer plugins in /app/lib/gstreamer-1.0/
0005  Found the following hardware decoding gstreamer plugin(s):
0006  1. vaapidecodebin
0007  Hardware decoding enabled.
0008  Session 677007046586700749 Opened '/home/.../test2.mix' (2 sources)
0009  OSC: Loaded 1 translation.
0010  OSC: Listening to UDP messages sent to 127.0.0.1:7000
0011  Font size 32
0012   Session '/home/.../test2.mix' loaded with 2 sources.
0013  MediaPlayer 681239528462868985 Opened 'video1.mp4' (H.264 (High Profile) 1920 x 1080)
0014  MediaPlayer 681239528462868985 Timeline [0 239979000000] 7192 frames, 0 gaps
0015  MediaPlayer 681239528490371715 Opened 'video2.mp4' (H.264 (High Profile) 1920 x 1080)
0016  MediaPlayer 681239528490371715 Timeline [0 276502000000] 8295 frames, 0 gaps
0017  MediaPlayer 681239528462868985 Uses vaapidecodebin decoding and OpenGL PBO texturing.
0018  Source 'video1' linked to MediaPlayer 681239528462868985.
0019  MediaPlayer 681239528490371715 Uses vaapidecodebin decoding and OpenGL PBO texturing.
0020  Source 'video2' linked to MediaPlayer 681239528490371715.
0021  Video Broadcast uses hardware-accelerated encoder (vaapih264enc)

I haven't had time to try latest source compile, but will give it a go when I can.

For completeness here are the errors on shutdown first time switching software decode to hardware:

0:00:27.022248023     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 6
0:00:27.022291576     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.023501455     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.023530198     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.023540148     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.023548309     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 2
0:00:27.023556807     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 0
0:00:27.023565666     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.023576663     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.023587860     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.023598100     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.023607519     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.024038217     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.024059254     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.024067886     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.024076076     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 4
0:00:27.024083675     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.024091224     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 2
0:00:27.024100008     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.024107592     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.024116867     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.024125940     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.024136054     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.024145115     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 5 is empty
0:00:27.024551004     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.024570030     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.024578356     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.024585919     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 6
0:00:27.024593193     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.024600215     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 4
0:00:27.024610277     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.024619533     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.024628537     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.024637158     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.024646460     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.024655456     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 5 is empty
0:00:27.025516760     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.025541005     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.025553131     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.025563088     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 2
0:00:27.025574485     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 0
0:00:27.025585522     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.025595742     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.025606604     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.025616159     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.025625250     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.026209724     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.026234143     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.026243290     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.026250698     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 4
0:00:27.026258224     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.026265084     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 2
0:00:27.026273913     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.026281692     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.026288593     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.026295296     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.026302108     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.026309614     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 5 is empty
0:00:27.026762728     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.026778818     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.026786643     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.026792481     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 6
0:00:27.026798520     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.026803980     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 4
0:00:27.026810813     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.026818227     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.026828069     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.026835056     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.026842219     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.026847867     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 5 is empty
0:00:27.027304217     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.027312774     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.027321059     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.027328181     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 2
0:00:27.027335964     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 0
0:00:27.027343502     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.027350386     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.027358099     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.027363433     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.027370107     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.027560218     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.027568164     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.027577449     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.027584566     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 4
0:00:27.027591411     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.027598089     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 2
0:00:27.027605707     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
0:00:27.027612547     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 1 is empty
0:00:27.027619341     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 2 is empty
0:00:27.027625894     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 3 is empty
0:00:27.027632582     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 4 is empty
0:00:27.027639509     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 5 is empty
0:00:27.027813153     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.027821396     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 5
0:00:27.027828334     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 3
0:00:27.027835295     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 4
0:00:27.027842998     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:2913:find_short_term_reference: found no short-term reference picture with PicNum = 1
0:00:27.027850558     2 0x7f78b4008aa0 ERROR                  vaapi gstvaapidecoder_h264.c:3119:exec_picture_refs_modification_1: list 0 entry 0 is empty
brunoherbelin commented 1 year ago

Thanks for your confirmation! Glad that the vaapi is detected with the Flatpak !

RKelln commented 1 year ago

I may have made some progress on this! I found some videos that open in the latest 0.8.0 self compiled version (with hardware acceleration on) and was able to compare to videos that don't work. The consistent difference is the colorspace:

Not working:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'black_2s.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf59.16.100
  Duration: 00:00:02.01, start: 0.000000, bitrate: 42 kb/s
  Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 25 kb/s, 30 fps, 30 tbr, 15360 tbn, 60 tbc (default)

While working videos all have yuv420p(tv, smpte170m) (but everything else the same).

I tried converting a non-working video to a working one, but that didn't resolve the issue, so that is very strange.

ffmpeg -i bt709_colorspace.mp4 -color_primaries smpte170m -color_trc smpte170m -colorspace smpte170m  smpte_colorspace.mp4

Also tried creating a smpte240m colorspace in kdenlive and that also doesn't load. Tried converting to different video formats, mov didn't work, but webp did load but the colors were damaged. HEVC did work even with bt709 colorspace! As did converting to avi (which seems to not know the colorspace and uses software decoding?)

ffmpeg -i input.mp4 -c:v libx265 -vtag hvc1 -c:a copy test_hevc.mp4

Not sure if there is anything in vimix that might account for that? I can try converting everything to hevc for now.