av1 encoding not working with kernel 6.3.13

[DatPat]

Describe The Bug I am running nyanmiasaka's preview version of jellyfin with experimental av1 encoding on an amd phoenix gpu. hevc support works, av1 does not.

va info: `# ./vainfo Trying display: drm libva info: VA-API version 1.18.0 libva info: Trying to open /usr/lib/jellyfin-ffmpeg/lib/dri/radeonsi_drv_video.so libva info: Found init function __vaDriverInit_1_18 amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description. If they do, bad things may happen! libva info: va_openDriver() returns 0 vainfo: VA-API version: 1.18 (libva 2.18.0) vainfo: Driver version: Mesa Gallium driver 23.2.0-devel for AMD Radeon Graphics (gfx1103_r1, LLVM 15.0.6, DRM 3.52, 6.3.13-Unraid) vainfo: Supported profile and entrypoints VAProfileH264ConstrainedBaseline: VAEntrypointVLD VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice VAProfileH264Main : VAEntrypointVLD VAProfileH264Main : VAEntrypointEncSlice VAProfileH264High : VAEntrypointVLD VAProfileH264High : VAEntrypointEncSlice VAProfileHEVCMain : VAEntrypointVLD VAProfileHEVCMain : VAEntrypointEncSlice VAProfileHEVCMain10 : VAEntrypointVLD VAProfileHEVCMain10 : VAEntrypointEncSlice VAProfileJPEGBaseline : VAEntrypointVLD VAProfileVP9Profile0 : VAEntrypointVLD VAProfileVP9Profile2 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointVLD VAProfileAV1Profile0 : VAEntrypointEncSlice VAProfileNone : VAEntrypointVideoProc

`

System (please complete the following information):

• Platform: AMD64
• Operating System: Unraid 6.12.3 with Kernel 6.3.13
• Virtualization: Docker
• Jellyfin Version: av1-enc prview
• Jellyfin-FFmpeg Version: 6.0-jf
• Hardware Acceleration: VAAPI
• CPU & GPU Model: AMD 7840hs

MediaInfo Happens on every file

FFmpeg Logs `ffmpeg version 6.0-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers built with gcc 12 (Debian 12.2.0-14) configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc libavutil 58. 2.100 / 58. 2.100 libavcodec 60. 3.100 / 60. 3.100 libavformat 60. 3.100 / 60. 3.100 libavdevice 60. 1.100 / 60. 1.100 libavfilter 9. 3.100 / 9. 3.100 libswscale 7. 1.100 / 7. 1.100 libswresample 4. 10.100 / 4. 10.100 libpostproc 57. 1.100 / 57. 1.100 amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description. If they do, bad things may happen! Input #0, matroska,webm, from 'file:/media/somefile.mkv': Metadata: encoder : libebml v1.4.4 + libmatroska v1.7.1 creation_time : 2023-05-14T17:51:26.000000Z Duration: 00:23:40.11, start: 0.000000, bitrate: 8111 kb/s Stream #0:0(spa): Subtitle: ass (default) Metadata: title : Castellano BPS : 109 DURATION : 00:23:31.990000000 NUMBER_OF_FRAMES: 274 NUMBER_OF_BYTES : 19313 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Stream #0:1(spa): Subtitle: ass Metadata: title : Latino BPS : 114 DURATION : 00:23:32.120000000 NUMBER_OF_FRAMES: 263 NUMBER_OF_BYTES : 20132 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Stream #0:2(eng): Subtitle: ass Metadata: title : English BPS : 128 DURATION : 00:23:31.910000000 NUMBER_OF_FRAMES: 354 NUMBER_OF_BYTES : 22621 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Stream #0:3: Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn Metadata: BPS : 7980889 DURATION : 00:23:40.045000000 NUMBER_OF_FRAMES: 34047 NUMBER_OF_BYTES : 1416652790 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Stream #0:4(jpn): Audio: aac (LC), 44100 Hz, stereo, fltp Metadata: BPS : 128000 DURATION : 00:23:40.109000000 NUMBER_OF_FRAMES: 61159 NUMBER_OF_BYTES : 22721748 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES Stream mapping: Stream #0:3 -> #0:0 (h264 (native) -> av1 (av1_vaapi)) Stream #0:4 -> #0:1 (copy) Press [q] to stop, [?] for help [hls @ 0x55d03b9d8000] Opening '/config/transcodes/20655d97ddcbaa4d57cdc981d4f6adb4-1.mp4' for writing Output #0, hls, to '/config/transcodes/20655d97ddcbaa4d57cdc981d4f6adb4.m3u8': Metadata: encoder : Lavf60.3.100 Stream #0:0: Vi log.txt deo: av1 (Main), vaapi(tv, bt709, progressive), 720x404 [SAR 404:405 DAR 16:9], q=2-31, 1372 kb/s, 23.98 fps, 24k tbn Metadata: BPS : 7980889 DURATION : 00:23:40.045000000 NUMBER_OF_FRAMES: 34047 NUMBER_OF_BYTES : 1416652790 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES encoder : Lavc60.3.100 av1_vaapi Stream #0:1(jpn): Audio: aac (LC), 44100 Hz, stereo, fltp Metadata: BPS : 128000 DURATION : 00:23:40.109000000 NUMBER_OF_FRAMES: 61159 NUMBER_OF_BYTES : 22721748 _STATISTICS_WRITING_APP: mkvmerge v76.0 ('Celebration') 64-bit _STATISTICS_WRITING_DATE_UTC: 2023-05-14 17:51:26 _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES frame= 0 fps=0.0 q=0.0 size=N/A time=00:00:00.00 bitrate=N/A speed= 0x
[hls @ 0x55d03b9d8000] Opening '/config/transcodes/20655d97ddcbaa4d57cdc981d4f6adb40.mp4' for writing [hls @ 0x55d03b9d8000] Opening '/config/transcodes/20655d97ddcbaa4d57cdc981d4f6adb41.mp4' for writing

[q] command received. Exiting.

[hls @ 0x55d03b9d8000] Opening '/config/transcodes/20655d97ddcbaa4d57cdc981d4f6adb42.mp4' for writing frame= 288 fps=0.0 q=-0.0 Lsize=N/A time=00:00:12.02 bitrate=N/A speed=29.4x
video:1958kB audio:200kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown`

Additional Context

[nyanmisaka]

@DatPat Can you test av1_vaapi encoder with this command?

/usr/lib/jellyfin-ffmpeg/ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc2=s=1280x720,format=nv12 -vf hwupload -c:v:0 av1_vaapi -rc_mode VBR -b:v 4M -maxrate 4M -bufsize 8M -profile:v:0 main -flags:v -global_header -f null -

[DatPat]

# /usr/lib/jellyfin-ffmpeg/ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc2=s=1280x720,format=nv12 -vf hwupload -c:v:0 av1_vaapi -rc_mode VBR -b:v 4M -maxrate 4M -bufsize 8M -profile:v:0 main -flags:v -global_header -f null - ffmpeg version 6.0-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers built with gcc 12 (Debian 12.2.0-14) configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc libavutil 58. 2.100 / 58. 2.100 libavcodec 60. 3.100 / 60. 3.100 libavformat 60. 3.100 / 60. 3.100 libavdevice 60. 1.100 / 60. 1.100 libavfilter 9. 3.100 / 9. 3.100 libswscale 7. 1.100 / 7. 1.100 libswresample 4. 10.100 / 4. 10.100 libpostproc 57. 1.100 / 57. 1.100 amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description. If they do, bad things may happen! Input #0, lavfi, from 'testsrc2=s=1280x720,format=nv12': Duration: N/A, start: 0.000000, bitrate: N/A Stream #0:0: Video: wrapped_avframe, nv12, 1280x720 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn Stream mapping: Stream #0:0 -> #0:0 (wrapped_avframe (native) -> av1 (av1_vaapi)) Press [q] to stop, [?] for help Output #0, null, to 'pipe:': Metadata: encoder : Lavf60.3.100 Stream #0:0: Video: av1 (Main), vaapi(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 4000 kb/s, 25 fps, 25 tbn Metadata: encoder : Lavc60.3.100 av1_vaapi frame= 0 fps=0.0 q=0.0 size= 0kB time=-577014:32:22.77 bitrate= -0.0kbitsframe=20505 fps=1366 q=-0.0 size=N/A time=00:13:40.20 bitrate=N/A speed=54.6x

it's running

[nyanmisaka]

Are you experiencing playback errors when using the web client? If yes please share the browser console output (F12). It seems that the AMD driver produces an av1 bitstream that browsers don't like.

[DatPat]

htmlVideoPlayer-plugin.33984355423e5d5c4cf3.chunk.js:2 Shaka: error code 3016 object Me value @ htmlVideoPlayer-plugin.33984355423e5d5c4cf3.chunk.js:2

I think is relevant

[nyanmisaka]

It does matter. I forgot to say that you only need to copy the log after clicking the play button.

[DatPat]

Media resource blob:http://10.0.0.84:8096/06e7d81e-cfef-4691-b2fa-9f045a516ab8 could not be decoded. index.html

This error message will be blank when privacy.resistFingerprinting = true. If it is really necessary, please add it to the whitelist in MediaError::GetMessage: NS_ERROR_DOM_MEDIA_METADATA_ERR (0x806e0006) - virtual RefPtr __cdecl mozilla::MP4Demuxer::Init(void): Parse MP4 metadata failed 4722.f96bd697d29d830e5706.chunk.js:2:204420

Shaka: error code 3016 object Object { severity: 2, category: 3, code: 3016, data: (3) […], handled: false } htmlVideoPlayer-plugin.33984355423e5d5c4cf3.chunk.js:2:16754 value http://10.0.0.84:8096/web/htmlVideoPlayer-plugin.33984355423e5d5c4cf3.chunk.js:2 e http://10.0.0.84:8096/web/htmlVideoPlayer-plugin.33984355423e5d5c4cf3.chunk.js:2 dispatchEvent http://10.0.0.84:8096/web/4722.f96bd697d29d830e5706.chunk.js:2 El http://10.0.0.84:8096/web/4722.f96bd697d29d830e5706.chunk.js:2 Ku http://10.0.0.84:8096/web/4722.f96bd697d29d830e5706.chunk.js:2

[DatPat]

This is all that was written to the console after hitting play

[nyanmisaka]

Thanks. Can you play the file encoded using these two command lines in the browser? Btw what's your the browser?

global_header_OFF.mp4

/usr/lib/jellyfin-ffmpeg/ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc2=s=1280x720,format=nv12 -vf hwupload -c:v:0 av1_vaapi -rc_mode VBR -b:v 4M -maxrate 4M -bufsize 8M -profile:v:0 main -force_key_frames:0 "expr:gte(t,0+n_forced*4)" -flags:v -global_header -vframes 1000 -y global_header_OFF.mp4

global_header_ON.mp4

/usr/lib/jellyfin-ffmpeg/ffmpeg -vaapi_device /dev/dri/renderD128 -f lavfi -i testsrc2=s=1280x720,format=nv12 -vf hwupload -c:v:0 av1_vaapi -rc_mode VBR -b:v 4M -maxrate 4M -bufsize 8M -profile:v:0 main -force_key_frames:0 "expr:gte(t,0+n_forced*4)" -flags:v +global_header -vframes 1000 -y global_header_ON.mp4

[DatPat]

it's firefox but I also have edge trying now

[DatPat]

global_header_OFF.mp4 will not play in firefox. global_header_ON.mp4 works

both files play on vlc

[DatPat]

both files play on edge, but transcoding playback will not work on edge

[nyanmisaka]

Well this seems to be a regression on server side which was caused by fixing other issue. I will rebuild a docker image for you to test.

[DatPat]

that's amazing, thanks so much

[nyanmisaka]

@DatPat You can try this now. I force pushed into the nyanmisaka/jellyfin:av1enc-preview https://hub.docker.com/layers/nyanmisaka/jellyfin/av1enc-preview/images/sha256-d641928b4a0065170d185d88f6308308b7642a520876e8047b01836f2f028d18?context=explore

[DatPat]

now it works!

[nyanmisaka]

Great! Actually I don't have the latest AMD APU platform to test the AV1 encoder. It would be great if you can share the encoding quality of it.

[DatPat]

how can I give you the information in a way that is useful?

so fps are very high (300+) but what I do notice is that there is a noticeable difference in quality between hevc and av1. AV1 is much cleaner than hevc video at the same bitrate. I would even say that AV1 much more in line with the output of my intel arc, than AMD's hevc implementation.

[DatPat]

sometimes it still gives me that error but very randomly. I am completely confused as to why

[DatPat]

okay, after I posted this it all magically started all working.

[DatPat]

EmeEncryptionSchemePolyfill: Already installed. 4722.f96bd697d29d830e5706.chunk.js:2:425502 McEncryptionSchemePolyfill: Already installed. 4722.f96bd697d29d830e5706.chunk.js:2:427216 Shaka: loaded manifest htmlVideoPlayer-plugin.33984355423e5d5c4cf3.chunk.js:2:18119 Active player: {"name":"Html Video Player","id":"htmlvideoplayer","playerName":"Html Video Player","playableMediaTypes":[false,true,false,false],"isLocalPlayer":true,"supportedCommands":["GoHome","GoToSettings","VolumeUp","VolumeDown","Mute","Unmute","ToggleMute","SetVolume","SetAudioStreamIndex","SetSubtitleStreamIndex","SetMaxStreamingBitrate","DisplayContent","GoToSearch","DisplayMessage","SetRepeatMode","SetShuffleQueue","PlayMediaSource","PlayTrailers","ToggleFullscreen","SetBrightness","SetAspectRatio","PlaybackRate"]} main.jellyfin.bundle.js:2:176502 SyncPlay WrapperFactory getWrapper: htmlvideoplayer main.jellyfin.bundle.js:2:310808 Requesting url without automatic networking: http://10.0.0.84:8096/Sessions/Playing main.jellyfin.bundle.js:2:871869 nowplaying event: playbackstart 9944.3751830cb9227bb61538.chunk.js:1:8379 nowplaying event: playbackstart 9944.3751830cb9227bb61538.chunk.js:1:9065 Requesting url without automatic networking: http://10.0.0.84:8096/Sessions/Playing/Progress main.jellyfin.bundle.js:2:871869 Media resource blob:http://10.0.0.84:8096/60c523e9-17b0-4cb3-b1b2-34b51056fd85 could not be decoded. 15 index.html Requesting url without automatic networking: http://10.0.0.84:8096/Sessions/Playing/Progress main.jellyfin.bundle.js:2:871869 [appRouter] "/video" route found

when it fails I get this, not sure why

[DatPat]

{ "severity": 2, "category": 3, "code": 3016, "data": [ 3, null, "NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005)" ], "handled": false }

[DatPat]

rebooting the server seems to fix it until it breaks again after a while.

[nyanmisaka]

how can I give you the information in a way that is useful?

so fps are very high (300+) but what I do notice is that there is a noticeable difference in quality between hevc and av1. AV1 is much cleaner than hevc video at the same bitrate. I would even say that AV1 much more in line with the output of my intel arc, than AMD's hevc implementation.

That’s quite enough. Let alone the hopeless AMD H264 encoder, AMD HEVC encoder still have obvious blocky patterns at relatively low bitrates and high motions senecios. Thanks to AV1's algorithm itself and AMD’s acquisition of Xilinx.

[nyanmisaka]

{ "severity": 2, "category": 3, "code": 3016, "data": [ 3, null, "NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005)" ], "handled": false }

When did this error occur? Beginning, middle or end? Did you seek the progress bar?

[DatPat]

{ "severity": 2, "category": 3, "code": 3016, "data": [ 3, null, "NS_ERROR_DOM_MEDIA_FATAL_ERR (0x806e0005)" ], "handled": false }

When did this error occur? Beginning, middle or end? Did you seek the progress bar?

right when I changed the bitrate to force transcoding

[DatPat]

tested more, starts right when I start playing every time if transcoding is active(that is if it happens).

[DatPat]

the av1 encoding seems to play much better on firefox than edge.