search

jellyfin / jellyfin-ffmpeg

FFmpeg for Jellyfin
https://jellyfin.org
Other
470 stars 124 forks source link

Transcoding w/ Subtitles Causes Entire System to Hang (Ryzen 3400G / Vega 11) #312

Open alexi741 opened 1 year ago

alexi741 commented 1 year ago

When attempting to transcode subtitles, GPU reports 100% usage, CPU utilization climbs unbounded, and the system hangs after a couple minutes. I must hard-reset the sever.

This makes it hard to watch my animes.

Kern.log https://pastebin.com/EywS6jE7

https://pastebin.com/EYWKTP0D

Highlights

[1208134.458864] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring comp_1.1.1 timeout, signaled seq=237, emitted seq=240
[1208134.459277] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process ffmpeg pid 601145 thread ffmpeg:cs0 pid 601147
[1208134.459647] amdgpu 0000:09:00.0: amdgpu: GPU reset begin!
[1208158.077783] clocksource: timekeeping watchdog on CPU3: hpet retried 2 times before success
[1208159.723714] watchdog: BUG: soft lockup - CPU#6 stuck for 26s! [kworker/u64:4:599072]

Media Info of the file https://pastebin.com/GqPWvvmD

Jellyfin Server Logs

[2023-01-30 19:39:14.943 -05:00] [INF] User policy for "alex". EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
[2023-01-30 19:39:14.943 -05:00] [INF] RemoteClientBitrateLimit: 16000000, RemoteIp: "192.168.1.243", IsInLocalNetwork: True
[2023-01-30 19:39:21.006 -05:00] [INF] Playback stopped reported by app "Jellyfin Roku" "1.6.3" playing "PROJECT APPLE". Stopped at "3000" ms
[2023-01-30 19:39:21.096 -05:00] [INF] User policy for "alex". EnablePlaybackRemuxing: True EnableVideoPlaybackTranscoding: True EnableAudioPlaybackTranscoding: True
[2023-01-30 19:39:21.096 -05:00] [INF] RemoteClientBitrateLimit: 16000000, RemoteIp: "192.168.1.243", IsInLocalNetwork: True
[2023-01-30 19:39:21.616 -05:00] [INF] Current HLS implementation doesn't support non-keyframe breaks but one is requested, ignoring that request
[2023-01-30 19:39:21.633 -05:00] [INF] "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-dump_attachment:t \"\" -y -i file:\"Spy x Family - S01E13.mkv\" -t 0 -f null null"
[2023-01-30 19:39:21.928 -05:00] [INF] ffmpeg attachment extraction completed for "/var/cache/jellyfin/attachments/88c9d7f26eb64f930f9e14e14bfb297f" to "/var/cache/jellyfin/attachments/88c9d7f26eb64f930f9e14e14bfb297f"
[2023-01-30 19:39:21.931 -05:00] [INF] "/usr/lib/jellyfin-ffmpeg/ffmpeg" "-analyzeduration 200M -ss 00:00:03.000 -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -i file:\"Spy x Family - S01E13.mkv\" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -codec:v:0 hevc_vaapi -tag:v:0 hvc1 -rc_mode VBR -b:v 4692002 -maxrate 4692002 -bufsize 9384004 -profile:v:0 main -force_key_frames:0 \"expr:gte(t,3+n_forced*3)\" -filter_complex \"alphasrc=s=1920x1080:r=10:start='00\:00\:03\.000',format=bgra,subtitles=f='Spy x Family - S01E13.mkv':si=0:alpha=1:sub2video=1:fontsdir='/var/cache/jellyfin/attachments/88c9d7f26eb64f930f9e14e14bfb297f',hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0,libplacebo=format=nv12:upscaler=none:downscaler=none,hwmap=derive_device=vaapi:reverse=1,format=vaapi\" -start_at_zero -codec:a:0 copy -copyts -avoid_negative_ts disabled -max_muxing_queue_size 2048 -f hls -max_delay 5000000 -hls_time 3 -hls_segment_type mpegts -start_number 1 -hls_segment_filename \"/var/lib/jellyfin/transcodes/b299e52bd9d339f2d981b48cacf9ffd6%d.ts\" -hls_playlist_type vod -hls_list_size 0 -y \"/var/lib/jellyfin/transcodes/b299e52bd9d339f2d981b48cacf9ffd6.m3u8\""
[2023-01-30 19:39:52.899 -05:00] [INF] Stopping ffmpeg process with q command for "/var/lib/jellyfin/transcodes/b299e52bd9d339f2d981b48cacf9ffd6.m3u8"
[2023-01-30 19:39:57.904 -05:00] [INF] Killing FFmpeg process for "/var/lib/jellyfin/transcodes/b299e52bd9d339f2d981b48cacf9ffd6.m3u8"
[2023-01-30 19:39:57.911 -05:00] [INF] Deleting partial stream file(s) "/var/lib/jellyfin/transcodes/b299e52bd9d339f2d981b48cacf9ffd6.m3u8"
[2023-01-30 19:39:59.416 -05:00] [INF] Playback stopped reported by app "Jellyfin Roku" "1.6.3" playing "PROJECT APPLE". Stopped at "0" ms

FFmpeg Logs https://pastebin.com/myX1RZ3j

Server System (please complete the following information):

Client System (please complete the following information):

alexi741 commented 1 year ago

It seems to be when subtitles get burned-in that the problem arises. I see it most often with Roku, but can replicate elsewhere.

For instance, the web app does not have a problem with the file in my report, but my system will hang if I attempt to play Shin Godzilla's Japanese Subtitles. (English subs are okay).

Text jellyfin/jellyfin#1
ID                                       : 4
Format                                   : ASS
Codec ID                                 : S_TEXT/ASS
Codec ID/Info                            : Advanced Sub Station Alpha
Duration                                 : 1 h 52 min
Bit rate                                 : 123 b/s
Count of elements                        : 1791
Compression mode                         : Lossless
Stream size                              : 102 KiB (0%)
Language                                 : English
Default                                  : Yes
Forced                                   : No

Text jellyfin/jellyfin#2
ID                                       : 5
Format                                   : PGS
Muxing mode                              : zlib
Codec ID                                 : S_HDMV/PGS
Codec ID/Info                            : Picture based subtitle format used on BDs/HD-DVDs
Duration                                 : 1 h 53 min
Bit rate                                 : 47.2 kb/s
Count of elements                        : 5183
Stream size                              : 38.2 MiB (0%)
Language                                 : Japanese
Default                                  : No
Forced                                   : No
nyanmisaka commented 1 year ago

The seems to be a kernel issue of amdgpu driver.

Can you try with the latest 6.1 or 6.2-rc?

alexi741 commented 1 year ago

I tried kernel 6.0.0-0.deb11.6-amd64 (bullseye-backports) without any luck. This weekend I'll upgrade to Debian Bookworm to see if the problem continues.

If it is a kernel / driver problem, what would the next steps be?

alexi741 commented 1 year ago

Can you try with the latest 6.1 or 6.2-rc?

I upgraded to these versions without luck.

MediaInfo https://pastebin.com/iFNS3Uyp

ffmpeg Log https://pastebin.com/pMLeifCV

kern.log https://pastebin.com/nXM6w6Tz

nyanmisaka commented 1 year ago

Can you try the latest release Jellyfin 10.8.9?

alexi741 commented 1 year ago

I downgraded to 10.8.9-1 and was able to transcode the videos.

That being said, I can't transcode HEVC now, which I believe is expected, because this version does not use Vulkan filtering.

It's interesting that I'm having an issue, as I see that this user has a similar iGPU (Vega 7) and can hard-code subs.

Jellyfin Log https://pastebin.com/222yFPp8

ffmpeg Log of SDR video https://pastebin.com/LeAafjzP

ffmpeg log of HEVC video https://pastebin.com/3xqdkkDk

dmesg while attempting to transcode HEVC video.

[80426.527959] ffmpeg[194669]: segfault at 60 ip 00007f4e3b90eb3d sp 00007ffdc5c52f70 error 4 in libLLVM-15.so.1[7f4e38e00000+677c000] likely on CPU 7 (core 3, socket 0)
[80426.527978] Code: ed 75 ea 31 c0 48 89 44 24 10 4d 85 f6 4c 89 74 24 40 4c 89 6c 24 38 0f 84 c3 00 00 00 4c 89 f7 e8 98 b3 28 fe 48 8b 54 24 48 <4c> 8b 6a 60 4d 85 ed 0f 84 bd 00 00 00 0f b6 4c 24 0f 40 88 cd 48
nyanmisaka commented 1 year ago

Both 2400G/3400G and 5600G are Vega but 5600G is 2nd Gen VCN. I used to report an issue for VCN1. Maybe there's something wrong with it again.

https://gitlab.freedesktop.org/mesa/mesa/-/issues/3117 I don't have 3400G anymore.

nyanmisaka commented 1 year ago

Can you try this command?

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"/Shin.Godzilla.mkv" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -codec:v:0 h264_vaapi -async_depth 1 -rc_mode VBR -b:v 15271028 -maxrate 15271028 -bufsize 30542056 -profile:v:0 high -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0,libplacebo=format=nv12:upscaler=none:downscaler=none,hwmap=derive_device=vaapi:reverse=1,format=vaapi" -start_at_zero -codec:a:0 libfdk_aac -ab 384000 -copyts -avoid_negative_ts disabled -f null -
alexi741 commented 1 year ago

Sure thing.

This command seems to be it, but it's inconsistent on the timing of the fault.

When I first ran it, ffmpeg froze around 15 seconds. After rebooting, it didn't freeze until about 46 seconds in. I even shut down other services like Jellyfin and radeontop to make sure that this command was the only thing using the iGPU.

dmesg

[Feb 6 13:33] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* ring comp_1.1.0 timeout, signaled seq=1720, emitted seq=1723
[  +0.000689] [drm:amdgpu_job_timedout [amdgpu]] *ERROR* Process information: process ffmpeg pid 202309 thread ffmpeg:cs0 pid 202310
[  +0.000633] amdgpu 0000:09:00.0: amdgpu: GPU reset begin!
$ /usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"./Shin.Godzilla.mkv" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -codec:v:0 h264_vaapi -async_depth 1 -rc_mode VBR -b:v 15271028 -maxrate 15271028 -bufsize 30542056 -profile:v:0 high -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0,libplacebo=format=nv12:upscaler=none:downscaler=none,hwmap=derive_device=vaapi:reverse=1,format=vaapi" -start_at_zero -codec:a:0 libfdk_aac -ab 384000 -copyts -avoid_negative_ts disabled -f null -
ffmpeg version 5.1.2-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[matroska,webm @ 0x5593b3643180] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (200000000) and 'probesize' (5000000) options
Input #0, matroska,webm, from 'file:./Shin.Godzilla.mkv':
  Metadata:
    title           : Shin Godzilla (2016) - 1080p
    encoder         : libebml v1.3.4 + libmatroska v1.4.5
    creation_time   : 2017-03-28T15:57:00.000000Z
  Duration: 01:59:50.19, start: 0.000000, bitrate: 15271 kb/s
  Chapters:
    Chapter #0:0: start 0.000000, end 11.136000
      Metadata:
        title           : Chapter 01
    Chapter #0:1: start 11.136000, end 23.523000
      Metadata:
        title           : Chapter 02
    Chapter #0:2: start 23.523000, end 27.318000
      Metadata:
        title           : Chapter 03
    Chapter #0:3: start 27.318000, end 31.614000
      Metadata:
        title           : Chapter 04
    Chapter #0:4: start 31.614000, end 890.931000
      Metadata:
        title           : Chapter 05
    Chapter #0:5: start 890.931000, end 1434.057000
      Metadata:
        title           : Chapter 06
    Chapter #0:6: start 1434.057000, end 1518.725000
      Metadata:
        title           : Chapter 07
    Chapter #0:7: start 1518.725000, end 1667.999000
      Metadata:
        title           : Chapter 08
    Chapter #0:8: start 1667.999000, end 1772.020000
      Metadata:
        title           : Chapter 09
    Chapter #0:9: start 1772.020000, end 1896.978000
      Metadata:
        title           : Chapter 10
    Chapter #0:10: start 1896.978000, end 2439.562000
      Metadata:
        title           : Chapter 11
    Chapter #0:11: start 2439.562000, end 2839.878000
      Metadata:
        title           : Chapter 12
    Chapter #0:12: start 2839.878000, end 2941.146000
      Metadata:
        title           : Chapter 13
    Chapter #0:13: start 2941.146000, end 3103.517000
      Metadata:
        title           : Chapter 14
    Chapter #0:14: start 3103.517000, end 3267.055000
      Metadata:
        title           : Chapter 15
    Chapter #0:15: start 3267.055000, end 3580.076000
      Metadata:
        title           : Chapter 16
    Chapter #0:16: start 3580.076000, end 3849.178000
      Metadata:
        title           : Chapter 17
    Chapter #0:17: start 3849.178000, end 4040.327000
      Metadata:
        title           : Chapter 18
    Chapter #0:18: start 4040.327000, end 4434.555000
      Metadata:
        title           : Chapter 19
    Chapter #0:19: start 4434.555000, end 5005.959000
      Metadata:
        title           : Chapter 20
    Chapter #0:20: start 5005.959000, end 5378.790000
      Metadata:
        title           : Chapter 21
    Chapter #0:21: start 5378.790000, end 5804.632000
      Metadata:
        title           : Chapter 22
    Chapter #0:22: start 5804.632000, end 6024.435000
      Metadata:
        title           : Chapter 23
    Chapter #0:23: start 6024.435000, end 6261.213000
      Metadata:
        title           : Chapter 24
    Chapter #0:24: start 6261.213000, end 6363.690000
      Metadata:
        title           : Chapter 25
    Chapter #0:25: start 6363.690000, end 6560.887000
      Metadata:
        title           : Chapter 26
    Chapter #0:26: start 6560.887000, end 6822.232000
      Metadata:
        title           : Chapter 27
    Chapter #0:27: start 6822.232000, end 7178.838000
      Metadata:
        title           : Chapter 28
    Chapter #0:28: start 7178.838000, end 7189.056000
      Metadata:
        title           : Chapter 29
    Chapter #0:29: start 7189.056000, end 7190.187000
      Metadata:
        title           : Chapter 30
  Stream #0:0: Video: h264 (High), yuv420p(progressive), 1920x804, SAR 1:1 DAR 160:67, 23.98 fps, 23.98 tbr, 1k tbn (default)
    Metadata:
      title           : 1080p - ZQ
      BPS             : 13166953
      BPS-eng         : 13166953
      DURATION        : 01:59:50.183000000
      DURATION-eng    : 01:59:50.183000000
      NUMBER_OF_FRAMES: 172392
      NUMBER_OF_FRAMES-eng: 172392
      NUMBER_OF_BYTES : 11834100999
      NUMBER_OF_BYTES-eng: 11834100999
      _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2017-03-28 15:57:00
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-03-28 15:57:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:1(jpn): Audio: dts (DTS), 48000 Hz, 3.1, fltp, 1536 kb/s (default)
    Metadata:
      title           : DTS 3.1
      BPS             : 1508999
      BPS-eng         : 1508999
      DURATION        : 01:59:50.187000000
      DURATION-eng    : 01:59:50.187000000
      NUMBER_OF_FRAMES: 674080
      NUMBER_OF_FRAMES-eng: 674080
      NUMBER_OF_BYTES : 1356248960
      NUMBER_OF_BYTES-eng: 1356248960
      _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2017-03-28 15:57:00
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-03-28 15:57:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:2(jpn): Audio: flac, 48000 Hz, stereo, s16
    Metadata:
      title           : FLAC 2.0
      BPS             : 546350
      BPS-eng         : 546350
      DURATION        : 01:59:50.186000000
      DURATION-eng    : 01:59:50.186000000
      NUMBER_OF_FRAMES: 84260
      NUMBER_OF_FRAMES-eng: 84260
      NUMBER_OF_BYTES : 491044904
      NUMBER_OF_BYTES-eng: 491044904
      _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2017-03-28 15:57:00
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-03-28 15:57:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:3(eng): Subtitle: ass, 1920x804 (default)
    Metadata:
      BPS             : 123
      BPS-eng         : 123
      DURATION        : 01:52:51.830000000
      DURATION-eng    : 01:52:51.830000000
      NUMBER_OF_FRAMES: 1791
      NUMBER_OF_FRAMES-eng: 1791
      NUMBER_OF_BYTES : 104635
      NUMBER_OF_BYTES-eng: 104635
      _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2017-03-28 15:57:00
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-03-28 15:57:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:4(jpn): Subtitle: hdmv_pgs_subtitle, 1920x804
    Metadata:
      BPS             : 47182
      BPS-eng         : 47182
      DURATION        : 01:53:03.651000000
      DURATION-eng    : 01:53:03.651000000
      NUMBER_OF_FRAMES: 5183
      NUMBER_OF_FRAMES-eng: 5183
      NUMBER_OF_BYTES : 40009074
      NUMBER_OF_BYTES-eng: 40009074
      _STATISTICS_WRITING_APP: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_APP-eng: mkvmerge v9.9.0 ('Pick Up') 64bit
      _STATISTICS_WRITING_DATE_UTC: 2017-03-28 15:57:00
      _STATISTICS_WRITING_DATE_UTC-eng: 2017-03-28 15:57:00
      _STATISTICS_TAGS: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
      _STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
  Stream #0:5: Attachment: ttf
    Metadata:
      filename        : framd.ttf
      mimetype        : application/x-truetype-font
  Stream #0:6: Attachment: ttf
    Metadata:
      filename        : timesbd.ttf
      mimetype        : application/x-truetype-font
Stream mapping:
  Stream #0:0 (h264) -> setparams:default (graph 0)
  Stream #0:4 (pgssub) -> scale:default (graph 0)
  format:default (graph 0) -> Stream #0:0 (h264_vaapi)
  Stream #0:1 -> #0:1 (dts (dca) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[matroska,webm @ 0x5593b3643180] sub2video: using 1920x804 canvas
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (High), vaapi(bt709, progressive), 1920x804 [SAR 1:1 DAR 160:67], q=2-31, 15271 kb/s, 23.98 fps, 23.98 tbn
    Metadata:
      encoder         : Lavc59.37.100 h264_vaapi
  Stream #0:1: Audio: aac, 48000 Hz, 4.0, s16, 384 kb/s (default)
    Metadata:
      encoder         : Lavc59.37.100 libfdk_aac
frame= 1044 fps= 47 q=-0.0 size=N/A time=00:00:43.88 bitrate=N/A speed=1.99x
Message from syslogd@flurry at Feb  6 14:26:05 ...
 kernel:[  360.268460] watchdog: BUG: soft lockup - CPU#3 stuck for 45s! [kworker/u64:10:274]
nyanmisaka commented 1 year ago

Can you downgrade jellyfin-ffmpeg5 to an old version 5.1.2-2? It uses older mesa 22.0.5 instead of the current 22.3.4.

alexi741 commented 1 year ago

I manually installed libllvm11 as a dependency, since it is not available in bookworm, and then installed jellyfin-ffmpeg5 5.1.2-2.

Same problem.

$ sudo dpkg -i libllvm11_11.0.1-2_amd64.deb jellyfin-ffmpeg5_5.1.2-2-bullseye_amd64.deb

$ /usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"./Shin.Godzilla.mkv" -autoscale 0 -map_metadata -1 -map_chapters -1 -threads 0 -map 0:0 -map 0:1 -codec:v:0 h264_vaapi -async_depth 1 -rc_mode VBR -b:v 15271028 -maxrate 15271028 -bufsize 30542056 -profile:v:0 high -force_key_frames:0 "expr:gte(t,0+n_forced*3)" -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0,libplacebo=format=nv12:upscaler=none:downscaler=none,hwmap=derive_device=vaapi:reverse=1,format=vaapi" -start_at_zero -codec:a:0 libfdk_aac -ab 384000 -copyts -avoid_negative_ts disabled -f null -
ffmpeg version 5.1.2-Jellyfin Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-libs=-lfftw3f --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-shared --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-static --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libmfx --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
[matroska,webm @ 0x5564ea2ab5c0] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (200000000) and 'probesize' (5000000) options
Input #0, matroska,webm, from 'file:/Shin.Godzilla.mkv':
  Metadata:
    title           : Shin Godzilla (2016) - 1080p
    encoder         : libebml v1.3.4 + libmatroska v1.4.5
    creation_time   : 2017-03-28T15:57:00.000000Z
  Duration: 01:59:50.19, start: 0.000000, bitrate: 15271 kb/s

*snip*   

Stream mapping:
  Stream #0:0 (h264) -> setparams:default (graph 0)
  Stream #0:4 (pgssub) -> scale:default (graph 0)
  format:default (graph 0) -> Stream #0:0 (h264_vaapi)
  Stream #0:1 -> #0:1 (dts (dca) -> aac (libfdk_aac))
Press [q] to stop, [?] for help
[matroska,webm @ 0x5564ea2ab5c0] sub2video: using 1920x804 canvas
Output #0, null, to 'pipe:':
  Metadata:
    encoder         : Lavf59.27.100
  Stream #0:0: Video: h264 (High), vaapi(bt709, progressive), 1920x804 [SAR 1:1 DAR 160:67], q=2-31, 15271 kb/s, 23.98 fps, 23.98 tbn
    Metadata:
      encoder         : Lavc59.37.100 h264_vaapi
  Stream #0:1: Audio: aac, 48000 Hz, 4.0, s16, 384 kb/s (default)
    Metadata:
      encoder         : Lavc59.37.100 libfdk_aac
frame= 1365 fps= 49 q=-0.0 size=N/A time=00:00:57.25 bitrate=N/A speed=2.05x
Message from syslogd@flurry at Feb  6 15:16:29 ...
 kernel:[ 2664.196389] watchdog: BUG: soft lockup - CPU#4 stuck for 45s! [kworker/u64:17:7511]
nyanmisaka commented 1 year ago

Please try these CLI and give some feedback.

decode + all filters

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"/Shin.Godzilla.mkv" -autoscale 0 -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0,libplacebo=format=nv12:upscaler=none:downscaler=none,hwmap=derive_device=vaapi:reverse=1,format=vaapi" -an -sn -f null -

decode + no reverse map

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"/Shin.Godzilla.mkv" -autoscale 0 -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0,libplacebo=format=nv12:upscaler=none:downscaler=none" -an -sn -f null -

decode + no placebo

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"/Shin.Godzilla.mkv" -autoscale 0 -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16[sub];[0:0]setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan[main];[main][sub]overlay_vulkan=eof_action=endall:shortest=1:repeatlast=0" -an -sn -f null -

decode + no overlay

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -hwaccel vaapi -hwaccel_output_format vaapi -autorotate 0 -canvas_size 1920x804 -i file:"/Shin.Godzilla.mkv" -autoscale 0 -vf "setparams=color_primaries=bt709:color_trc=bt709:colorspace=bt709,scale_vaapi=format=bgra:extra_hw_frames=32,hwmap=derive_device=vulkan,format=vulkan" -an -sn -f null -

transfer only

/usr/lib/jellyfin-ffmpeg/ffmpeg -analyzeduration 200M -init_hw_device vaapi=va:/dev/dri/renderD128 -filter_hw_device va -autorotate 0 -canvas_size 1920x804 -i file:"/Shin.Godzilla.mkv" -autoscale 0 -filter_complex "[0:4]scale=s=1920x804:flags=fast_bilinear,format=bgra,hwupload=derive_device=vulkan:extra_hw_frames=16,format=vulkan" -vn -an -sn -f null -
alexi741 commented 1 year ago

I upgraded back to ffmpeg 5.1.2-7

decode + all filters

GPU reset. System hangs.

decode + no reverse map

Transcoded the entire file without issue.

decode + no placebo

GPU reset. System hangs.

decode + no overlay

Transcoded the entire file without issue.

transfer only

ffmpeg complained a little bit, sending lots of error messages like this. Otherwise it completed without issue. Application provided invalid, non monotonically increasing dts to muxer in stream 0: 169979 >= 169979

alexi741 commented 1 year ago

So I'm thinking of upgrading to a Ryzen Pro 5650g, which has VCN 2, to get away from this problem.

@nyanmisaka, would you mind sharing your thoughts? If this looks like a kernel/driver issue that won't be solved anytime soon, would you recommend changing processors or could I disable reverse mapping somehow?

nyanmisaka commented 1 year ago

@alexi741 I don't have such issue on my Ryzen 7 5700G from my last testing. Also verified with VEGA64, Intel HD630, Xe and Arc A380 with the same command.

The reverse mapping is essential for zero-copy transcoding. So its more like the issue is caused by the VCN1 KMD. You can try this 5.1.2-8 build that has some vaSyncSurface tweaks in it. I don't know it it helps.

jellyfin-bot commented 1 year ago

This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.

If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or master branch, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.

This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.

nyanmisaka commented 1 year ago

not stale @jellyfin-bot

mushkevych commented 11 months ago

I have experienced a similar issue, when a selection of subtitles from Roku app causes the Jellyfin server to use 100% of all the CPUs and sporadically hang. I tried limiting bitrate and set transcoding path to a RAM-disk as following in /etc/fstab: transcoderamdisk /tmp/ramdisk tmpfs defaults,size=16G,x-gvfs-show 0 0

and volumes in jellyfin Dockerfile stack: - /tmp/ramdisk:/data/transcodes

however to no avail.

My setup: