search

mltframework / mlt

MLT Multimedia Framework
https://www.mltframework.org
GNU Lesser General Public License v2.1
1.48k stars 314 forks source link

use VAAPI to decode AVCHD (and other formats) #228

Closed rugubara closed 7 years ago

rugubara commented 7 years ago

Hello, Although you have https://www.mltframework.org/faq/#is-va-api-supported in your FAQ, I would like to challenge the pre-requisites of this decision.

  1. There are GLSL based effects (based on movit). They run on intel haswell GPU pretty well.
  2. My haswell i7 laptop runs ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_format vaapi -i input.MTS -an -vf 'format=nv12|vaapi,hwupload,scale_vaapi=w=1280:h=720:format=yuv420p,hwdownload,format=nv12' -f null - at 18x speed and it runs CPU-based ffmpeg -i input.MTS -f null -at 7x speed. The input file is FullHD x264 high profile footage from a prosumer sony camcorder.

When I try to do 2 track edits from 2 cameras in kdenlive (all decoding is CPU based, track compositing, GLSL based wipes), I never get above 7 fps when there are 2 video tracks active at the same time. When there are 3 tracks active - it is 1 fps slide show. 1 track plays at 30 fps fine. kdenlive is configured for xgl consumer.

When I decode and encode back, scaling in VAAPI in between, I can do it at 14x. ffmpeg -vaapi_device /dev/dri/renderD128 -hwaccel vaapi -hwaccel_output_forma t vaapi -i input.MTS -an -vf "format=nv12|vaapi,hwupload,scale_vaapi=w=1280:h=720" -c:v h264_vaapi -f mp4 /dev/null

Taking the huge performace impact of using VAAPI, even when I download the processed surfaces to the main memory, I believe enabling VAAPI is worth doing now.

ddennedy commented 7 years ago

This is only for tracking defects; not feature requests.

ericfont commented 6 years ago

FYI, as a workaround, can output mlt video as "vcodec=rawvideo", which will produce a huge file (without wasting cpu on compression), and then can run the ffmpeg vaapi command with that raw video as input.

jonassmedegaard commented 3 years ago

see also #621

ddennedy commented 3 years ago

This works now with caveats. To use, specify the file name like fs:///full/path/to/video.mp4?hwaccel=vaapi

hwaccel accepts one the following values:

fs:// is needed because MLT currently accepts '?' as a file name character without escape. We hope to change that for next major version currently in development and require '?' in a filename to be escaped with a '\' and subsequently drop the fs:// protocol.

There is currently no plan to make hwaccel work more automatically and with software fallback. Apps adopting this will have to fully manage all aspects of its usage. That also includes resource limitations (codec profiles, resolution, number of active decoders, etc.) In the context of melt, that management falls back to the user.

jonassmedegaard commented 3 years ago

very nice - thanks to those who made it happen!