Support high-resolution videos in video_player

[Kofhein]

Currently when playing videos higher than 1080p 30fps(for example 1080p 60fps) it cause player to lose sound and play slower(1080p 60 fps plays like 30), the higher resolution - slower playback is.

• On 4K it looks like slow motion video.
• HW acceleration is present ( VAAPI ), also tried separate pipeline without sound, just video - same result. Pipeline: filesrc ! qtdemux ! queue ! h264parse ! vaapih264dec ! vaapipostproc ! fakesink
• When sound disappears underflow happens: WARN pulse pulsesink.c:703:gst_pulsering_stream_underflow_cb:<audiosink-actual-sink-pulse> Got underflow
• There's no resources starvation CPU/GPU

Tests were performed with big-buck samples:

• 1080p 30fps(no issues)
• 1080p 60 fps(video playback looks like 30 fps or slower, sound disappears once introduction animation ends)
• 4K 30 fps (super slow video, same issue with sound )

Tested on Laptop with Ubuntu 20.04, Intel HD Graphics ( Latest drivers as well as VAAPI lib were built from source ) Playback is fine in separate window if don't provide videosink to playbin.

[HidenoriMatsubayashi]

This issue is related to #64.

[Kofhein]

@HidenoriMatsubayashi , I've performed more testing and checks, basically flag "qos" is set to FALSE, if set it to TRUE - sound issue is being resolved and 1080p 60 fps plays fine, also it improves playback speed for 4K

[Kofhein]

Also with some more digging next results were revealed - on laptop/Desktop(I know that target of this platform is embedded devices), but still - currently better performance will be if not use HW acceleration as it cause very poor performance operation - copying GPU buffers into system buffers and on PC it's actual bottleneck.

Here's results of measuring execution time of calling gst_buffer_extract on 4K 30fps without HW (Playback is smooth and normal) Time measured: 0.00904538 seconds (9ms) Time measured: 0.0134925 seconds (13ms) Time measured: 0.0245555 seconds (24ms) Time measured: 0.0124023 seconds (12ms) Time measured: 0.0209218 seconds (20ms) Time measured: 0.0139477 seconds (13ms) Time measured: 0.0101438 seconds (10ms) Time measured: 0.0148435 seconds (14ms) Time measured: 0.0162012 seconds (16ms) Time measured: 0.0172743 seconds (17ms) Time measured: 0.020268 seconds (20 ms) Time measured: 0.0113564 seconds (11 ms)

And this is when using HW and enabling GST_DEBUG (Playback is slow), on each buffer we get Performance warning. 0:00:28.079837425 16844 0x7f2164015aa0 INFO GST_PERFORMANCE gstvaapipluginbase.c:1589:gst_vaapi_plugin_copy_va_buffer: copying VA buffer to system memory buffer Time measured: 0.0151221 seconds (15ms) 0:00:28.148588527 16844 0x7f2164015aa0 INFO GST_PERFORMANCE gstvaapipluginbase.c:1589:gst_vaapi_plugin_copy_va_buffer: copying VA buffer to system memory buffer Time measured: 0.0163416 seconds (16ms) 0:00:28.195588318 16844 0x7f2164015aa0 INFO GST_PERFORMANCE gstvaapipluginbase.c:1589:gst_vaapi_plugin_copy_va_buffer: copying VA buffer to system memory buffer Time measured: 0.0128311 seconds (12ms) 0:00:28.240566048 16844 0x7f2164015aa0 INFO GST_PERFORMANCE gstvaapipluginbase.c:1589:gst_vaapi_plugin_copy_va_buffer: copying VA buffer to system memory buffer Time measured: 0.0183564 seconds (18ms) 0:00:28.289494000 16844 0x7f2164015aa0 INFO GST_PERFORMANCE gstvaapipluginbase.c:1589:gst_vaapi_plugin_copy_va_buffer: copying VA buffer to system memory buffer Time measured: 0.0138344 seconds (13ms)

VA-API plugin warns about performance issue and between each of this warning from 50 to 70 ms could pass. Without GPU buffers hw acceleration isn't useful :(

[Kofhein]

@HidenoriMatsubayashi , Hi, I found solution for desktop, I think it's pretty much HW dependent - in gstreamer there're plugins that are capable to write into DMABuf, it's explicitly shown in caps description video/x-raw(memory:DMABuf), but one thing is that everything should be wired with correct plugins, otherwise even with this caps it won't show any performance. So I'll describe my case with intel vaapi - currently in gstreamer there're 2 plugin packs vaapi and va, in vaapi - vaapipostproc is not capable to write into DMABuf mem, but vapostproc is and here comes pitfall - va plugins rank is none, so internally playbin generates vaapi based decoding, which leads to performance loss due to inability to write into DMABufs. Also reason I had that GST_PERFORMANCE message was exactly because I didn't changed caps from "video/x-raw, format=RGBA". VAAPI based plugins provides also ability to use vaapi mem types such as VASurface and VAMemory, but they unable to properly handle Album orientation of video - only landscape will be good if this type of mem is used in the output caps. Now I'm able to play 4K video with HW acceleration.