[Feature Request] AV1 (aomenc) Support

[mbattista]

With chrome 90 I did some tests with the AV1 real time encoder and it should be fine on most systems. (Screenshare 1600x900 with ~30 fps on an 4500U was possible)

Would you be fine with an PR where aomenc is used? The main problem would be that even a change to an "unstable" apt to install gtreamer 1.18 instead of 1.16 would not be enough, since the deadline / realtime flags are not available (yet).

This would propably add a lot of extra code and changes, since I think about using https://aomedia.googlesource.com/aom/+/webm_nextgenv2/aomenc.c directly. Hence why I ask beforehand if you think this is something you would approve.

[m1k1o]

I would not like to introduce unstable code to main branch and casual day-to-day usage. But I am fully open for experimenting with new features even if they are now WIP, unstable or just PoC. And also willing to help with analysing / implementing.

I would suggest to add a new Dockerfile e.g. Dockerfile.unstable or Dockerfile.av1 to try this out. What do you say?

[m1k1o]

If you want to completly change the way how it is used by Go, then we can either think about having it in different branch or try to scope it to a single module that can be either conditionally compiled to the project or has just basic switch, whether is used or not.

[mbattista]

I would not like to introduce unstable code to main branch and casual day-to-day usage. But I am fully open for experimenting with new features even if they are now WIP, unstable or just PoC. And also willing to help with analysing / implementing.

On the website gstreamer anounces that 1.18.4 is a stable version. Similar to nodejs an install via apt stable delivers a safe version which is 1.16, while the the 1.18 version is in apt unstable. For h264, vp8, vp9 and opus it did not really matter, since in those encoder not much did change in the last years (at least to my knowlege). For AV1 every version gives performance upgrades.

If you want to completly change the way how it is used by Go, then we can either think about having it in different branch or try to scope it to a single module that can be either conditionally compiled to the project or has just basic switch, whether is used or not.

To be honest, I still have no idea, how to implement it. Since there are a lot of hurdles right now. I just wanted to ask before I start, since it could bring some changes under the hood, e.g. using ffmpeg instead of gstreamer or a custom made gstreamer plugin, which will be added in the dockerfile.

[m1k1o]

I am fine with it even having it in the main branch. But I would prefer to have every new feature as addon/separate module and being able to switch between them in config (e.g., if we want to use gstreamer or ffmpeg). Also, I am fine with having custom Dockerfile for this experiment.

I would say, if you want to implement it and try it, go ahead. You do not need to take care about structuring the code to be completely split to custom module, since that is extra overhead while implementing an PoC. I can update it afterwards and add your changes to neko's codebase without excluding any existing functionality.

[m1k1o]

Gstreamer 1.18 is already stable and we are using it now:

root@neko:/# dpkg -l | grep gstreamer
ii  gir1.2-gstreamer-1.0:amd64             1.18.4-2.1                     amd64        GObject introspection data for the GStreamer library
ii  gstreamer1.0-plugins-bad:amd64         1.18.4-3                       amd64        GStreamer plugins from the "bad" set
ii  gstreamer1.0-plugins-base:amd64        1.18.4-2                       amd64        GStreamer plugins from the "base" set
ii  gstreamer1.0-plugins-good:amd64        1.18.4-2+deb11u1               amd64        GStreamer plugins from the "good" set
ii  gstreamer1.0-plugins-ugly:amd64        1.18.4-2                       amd64        GStreamer plugins from the "ugly" set
ii  gstreamer1.0-pulseaudio:amd64          1.18.4-2+deb11u1               amd64        GStreamer plugin for PulseAudio (transitional package)
ii  gstreamer1.0-vaapi:amd64               1.18.4-2                       amd64        VA-API plugins for GStreamer
ii  libgstreamer-gl1.0-0:amd64             1.18.4-2                       amd64        GStreamer GL libraries
ii  libgstreamer-plugins-bad1.0-0:amd64    1.18.4-3                       amd64        GStreamer libraries from the "bad" set
ii  libgstreamer-plugins-base1.0-0:amd64   1.18.4-2                       amd64        GStreamer libraries from the "base" set
ii  libgstreamer-plugins-base1.0-dev:amd64 1.18.4-2                       amd64        GStreamer development files for libraries from the "base" set
ii  libgstreamer1.0-0:amd64                1.18.4-2.1                     amd64        Core GStreamer libraries and elements
ii  libgstreamer1.0-dev:amd64              1.18.4-2.1                     amd64        GStreamer core development files

[ehfd]

AOM and rav1e are very slow, but I have heard good news about svtav1enc. Might work as the SW encoder of choice.