Add v4l2_codec2 and ffmpeg_codec2

[mikegapinski]

v4l2_codec2 provides hardware accelerated h264 encoding ffmpeg_codec2 provides hardware accelerated encoding for h264 and h265. Various other formats via software decoding

TODO: switch to mainline ffmpeg via aospext with patches from Jernej, fork ffmpeg_codec2 from raspberry-vanilla

[rsglobal]

@mikegapinski ,

Thank you for this PR

Could you share your experience on how to test codecs? Making a Wiki page someday would be nice.

[rsglobal]

From my experience CTS and VTS commands are:

atest CtsVideoTestCases
atest VtsHalMediaC2V1_0TargetVideoEncTest VtsHalMediaC2V1_0TargetVideoDecTest VtsHalMediaC2V1_0TargetComponentTest

But some methodology for measuring codecs performance would be nice to have.

[mikegapinski]

@mikegapinski ,

Thank you for this PR

Could you share your experience on how to test codecs? Making a Wiki page someday would be nice.

I usually just do a BigBuckBunny in a few formats and observe the CPU usage - measuring codecs on Tesla Android is not a good idea because I run the virtual display all the time and results vary. At least they used to on my current public build without mjpeg acceleration

[mikegapinski]

@rsglobal if we are doing ffmpeg with aospext should I refactor this PR to only include the v4l2_codec2 for h264 encoding(scrcpy etc)?

Or you can ping me when ffmpeg is ready on your branch and I'll rebase on top it with my changes

[rsglobal]

@rsglobal if we are doing ffmpeg with aospext should I refactor this PR to only include the v4l2_codec2 for h264 encoding(scrcpy etc)?

Or you can ping me when ffmpeg is ready on your branch and I'll rebase on top it with my changes

My plan is to enable ffmpeg-codec2 for software codecs for now and apply it to all GloDroid devices. This requires modification of ffmpeg-codec2 to support meson.build and use aospext for building it (otherwise using ffmpeg-installed headers is not possible), So it will take some time. After that, we will add patches for h265 on top.

As for v4l2_codec2 I have no plans for it, so if you can split it from this repoPR, we can land it ASAP. It would be nice to put it as a part of gd_rpi4 device tree instead of glodroid configuration patches if possible. E.g. into .../gd_rpi4/codecs/ directory.

[mikegapinski]

@rsglobal if we are doing ffmpeg with aospext should I refactor this PR to only include the v4l2_codec2 for h264 encoding(scrcpy etc)? Or you can ping me when ffmpeg is ready on your branch and I'll rebase on top it with my changes

My plan is to enable ffmpeg-codec2 for software codecs for now and apply it to all GloDroid devices. This requires modification of ffmpeg-codec2 to support meson.build and use aospext for building it (otherwise using ffmpeg-installed headers is not possible), So it will take some time. After that, we will add patches for h265 on top.

As for v4l2_codec2 I have no plans for it, so if you can split it from this repo, we can land it ASAP. It would be nice to put it as a part of gd_rpi4 device tree instead of glodroid configuration patches if possible. E.g. into .../gd_rpi4/codecs/ directory.

OK. Understood. What I recommend at this state is leaving this PR unmerged as it is now. Anyone can just merge the branch locally and have hardware accelerated codecs in the best/only available form. I'll be using it in the Tesla Android builds until a better alternative emerges.

I will create another PR that adds v4l2 to rpi4 device that could be merged right now. This would give the GloDroid rpi4 builds with h264 encode/decode on top of your ffmpeg implementation with software decoders. The changes hwaccel in ffmpeg won't be happening on device level so no changes would be required here, once the patches show up it'll simply start working