VAAPI and AMD AMF hardware transcoding doesn't work without these updates

[Protektor-Desura]

Okay I figured out how to get VAAPI work in Jellyfin. There need to be some changes to the Jellyfin docker. You can not do VAAPI transcoding on Linux without Mesa 20.1+. The Mesa 20.1+ update must be installed in the docker image. I added this to docker and it started working for my RX550 video card.

apt-get update && apt-get upgrade -y
apt-get install -y gpg
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys F63F0F2B90935439
echo "deb http://ppa.launchpad.net/kisak/kisak-mesa/ubuntu bionic main" > /etc/apt/sources.list.d/kisak-ubuntu-kisak-mesa-bionic.list
apt-get update
apt-get upgrade -y

I also installed amdgpu-pro in the host OS as well. I suspect that Emby docker needs the same exact changes as well. This is needed for AMD AMF as well. Plus for AMD AMF you must install amdgpu-pro on the host OS in Linux. Open source only does decoding on AMD AMF and not sure it does any HEVC. Thus the need for amdgpu-pro for AMD AMF transcoding.

All of this is required for HEVC transcoding. If you don't do this you are limited to H.264 and older transcoding.

[jorritsmit]

i had vaapi decoding working with mesa 20.1 and knew that amdgpu-pro was needed for AMF decoding but never got it to work. Did you just install amdgpu-pro on the host? not in the docker?

[Protektor-Desura]

Correct you just install amdgpu-pro on the host not inside the docker.

[Legogris]

An alternative to get the same result:

add-apt-repository -y ppa:oibaf/graphics-drivers && \
apt-get update && \
apt-get dist-upgrade -y

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Legogris]

While this may be seen as a duplicate of https://github.com/linuxserver/docker-jellyfin/issues/38, I guess one of them should remain open?

[Protektor-Desura]

I've gotten the impression that the devs have no plans to fix this bug.

[aptalca]

This is not a bug

[Protektor-Desura]

Failure to work as it should given the instructions on the Jellyfin website makes it a bug. Failure to work at all as released is a bug.

[PascalMinder]

i had vaapi decoding working with mesa 20.1 and knew that amdgpu-pro was needed for AMF decoding but never got it to work. Did you just install amdgpu-pro on the host? not in the docker?

How did you get VAAPI working?

[jorritsmit]

@PascalMinder by installing mesa 20.1 in the docker container. This package is not yet present in the ubuntu repositories so you have to install it via a ppa. i use oibafs:

add-apt-repository ppa:oibaf/graphics-drivers -y
apt update -y
apt upgrade -y

Note: you have to do this every time you restart the container. Check here what version the ubuntu repo is currently on (almost there)

[PascalMinder]

@PascalMinder by installing mesa 20.1 in the docker container. This package is not yet present in the ubuntu repositories so you have to install it via a ppa. i use oibafs:

add-apt-repository ppa:oibaf/graphics-drivers -y
apt update -y
apt upgrade -y

Note: you have to do this every time you restart the container. Check here what version the ubuntu repo is currently on (almost there)

okay thanks. I'll try it out.

[PascalMinder]

@PascalMinder by installing mesa 20.1 in the docker container. This package is not yet present in the ubuntu repositories so you have to install it via a ppa. i use oibafs:

add-apt-repository ppa:oibaf/graphics-drivers -y
apt update -y
apt upgrade -y

Note: you have to do this every time you restart the container. Check here what version the ubuntu repo is currently on (almost there)

I might try to create a docker-mode with this changes on Friday if possible. So the changes would persist over reboot and any container update.

[PascalMinder]

So I created a mod and started a video decoding it. When running the following command grep -A2 'Stream mapping:' Log_2.txt against the log file I get the following output

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_vaapi))
  Stream #0:1 -> #0:1 (eac3 (native) -> mp3 (libmp3lame))

Does this now mean, the steam is encoded using the GPU or is it still on the CPU?

[jorritsmit]

Yes encode with gpu. h264_vaapi

[seanmikhaels]

I was able to get it to work with VAAPI by installing AMD Pro drivers on both host and inside docker. However, now i notice that loading more than one transocde at a time crashes the entire system. Seems like a kernel issues but not 100% sure. Dmesg shows ring timeout and failing gpu reset. I'm using 20.1.5 mesa included in latest gfx drivers, and Kernel 5.9.

[PascalMinder]

I created an image based on the container here. https://github.com/PascalMinder/docker-mods/tree/jellyfin-amd

For me it seems to work with vaapi but I have not tested it with 2 streams.

[nyanmisaka]

I think the Ubuntu focal based image now have an upgraded Mesa VAAPI driver. mesa-va-drivers (20.2.6-0ubuntu0.20.04.1)

[aptalca]

Here's a new mod for installing amd drivers thanks to @PascalMinder https://mods.linuxserver.io/?mod=jellyfin

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[tmplt]

VAAPI works with the above mod and amdgpu loaded for the card on the host. amdgpu-pro is not a requirement. Verify with lspci:

$ lspci -nnk
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev ef)
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:0b31]
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu

If you use an older card radeon may be used instead. This will cause decoding issues.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[j0nnymoe]

Closing as we have a mod for this.