HaveAGitGat / Tdarr

Tdarr - Distributed transcode automation using FFmpeg/HandBrake + Audio/Video library analytics + video health checking (Windows, macOS, Linux & Docker)
2.72k stars 89 forks source link

Tdarr keeps crashing host #970

Closed exodus454 closed 2 weeks ago

exodus454 commented 2 months ago

Describe the bug Tdarr keeps coredumping his podman container. Probably related to the other memory leak bug. Happens somewhat randomly. I've seen logs mentioning ccextractor and Tdarr_server, tdarr-ffmpeg.

It actually got MUCH worse with the latest (6.7.7-100.fc38) kernel. I had been running the LTS (6.1? maybe 6.3) becuase the ZFS modules weren't compatible. When I switched about a week ago the issue started getting much worse.

For now I'm trying running it with --memory 2G to try to control it. Unrestricted it was 6-8G at the very least. This machine has 32G of ram, zram-swap and 190G ssd backing device, so free ram is always 1-2G - ie hard to spot memory leaks.

I attached all the coredump info, let me know if anything else would be useful.

To Reproduce Steps to reproduce the behavior: Run in podman container

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

systemd-coredump history March 5, 2024 3:43 AM Process 356909 (snapd) of user 0 dumped core. Module snapd from rpm snapd-2.61.2-0.fc38.x86_64 Stack trace of thread 371736: #0 0x000056450f0f

systemd-coredump 12:08 AM Process 187309 (Tdarr_Server) of user 0 dumped core. Module /usr/bin/node from rpm nodejs18-18.19.0-1.fc38.x86_64 Stack systemd-coredump March 4, 2024 Reboot

systemd-coredump Process 187309 (Tdarr_Server) of user 0 dumped core. Module /usr/bin/node from rpm nodejs18-18.19.0-1.fc38.x86_64 Stack trace of thread 380: #0 0x00007fe40289e00b n/a (/usr/lib/x86_64-linux-gnu/libc-2.31.so + 0x4300b) #1 0x0000000000a94834 n/a (/usr/bin/node + 0x694834) #2 0x0000000000d647c0 n/a (/usr/bin/node + 0x9647c0) #3 0x0000000000d64b67 n/a (/usr/bin/node + 0x964b67) #4 0x0000000000f42265 n/a (/usr/bin/node + 0xb42265) #5 0x0000000000f5474d n/a (/usr/bin/node + 0xb5474d) #6 0x0000000000f2ee4e n/a (/usr/bin/node + 0xb2ee4e) #7 0x0000000000f30217 n/a (/usr/bin/node + 0xb30217) #8 0x0000000000f113ea n/a (/usr/bin/node + 0xb113ea) #9 0x00000000012d674f n/a (/usr/bin/node + 0xed674f) #10 0x00000000017035b9 n/a (/usr/bin/node + 0x13035b9) #11 0x0000000006e905ce n/a (n/a + 0x0) #12 0x0000000006e92b54 n/a (n/a + 0x0) #13 0x00000000016bb2ff n/a (/usr/bin/node + 0x12bb2ff) #14 0x0000000001753bb1 n/a (/usr/bin/node + 0x1353bb1) #15 0x00000000016aca7b n/a (/usr/bin/node + 0x12aca7b) #16 0x0000000001685b43 n/a (/usr/bin/node + 0x1285b43) #17 0x0000000000ea548a n/a (/usr/bin/node + 0xaa548a) #18 0x0000000000ea6090 n/a (/usr/bin/node + 0xaa6090) #19 0x0000000000ea679a n/a (/usr/bin/node + 0xaa679a) #20 0x0000000000ed3c9a n/a (/usr/bin/node + 0xad3c9a) #21 0x0000000000ed4072 n/a (/usr/bin/node + 0xad4072) #22 0x0000000000ac4c12 n/a (/usr/bin/node + 0x6c4c12) #23 0x0000000000ac50f6 n/a (/usr/bin/node + 0x6c50f6) #24 0x0000000000ad722d n/a (/usr/bin/node + 0x6d722d) #25 0x0000000000b88c32 n/a (/usr/bin/node + 0x788c32) #26 0x0000000000b8c6d1 n/a (/usr/bin/node + 0x78c6d1) #27 0x0000000000b80974 n/a (/usr/bin/node + 0x780974) #28 0x000000000166224d n/a (/usr/bin/node + 0x126224d) #29 0x0000000001666a36 n/a (/usr/bin/node + 0x1266a36) #30 0x0000000001678f74 n/a (/usr/bin/node + 0x1278f74) #31 0x000000000166739e n/a (/usr/bin/node + 0x126739e) #32 0x0000000000ac5a6d n/a (/usr/bin/node + 0x6c5a6d) #33 0x0000000000bc8d14 n/a (/usr/bin/node + 0x7c8d14) #34 0x0000000000b3ec18 n/a (/usr/bin/node + 0x73ec18) #35 0x0000000000b4272f n/a (/usr/bin/node + 0x74272f) #36 0x00007fe40287f083 n/a (/usr/lib/x86_64-linux-gnu/libc-2.31.so + 0x24083) ELF object binary architecture: AMD x86-64 CODE_FILE ../src/coredump/coredump.c CODE_FUNC submit_coredump CODE_LINE 1053 COREDUMP_CGROUP /machine.slice/libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope/container COREDUMP_CMDLINE Tdarr_Server COREDUMP_COMM Tdarr_Server COREDUMP_CONTAINER_CMDLINE /usr/bin/conmon --api-version 1 -c d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea -u d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea -r /usr/bin/crun -b /var/lib/containers/ssd0/storage/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata -p /var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/pidfile -n tdarr --exit-dir /run/libpod/exits --full-attach -s -l journald --log-level warning --syslog --log-size-max 10000000 --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/oci-log --conmon-pidfile /var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/ssd0/storage --exit-command-arg --runroot --exit-command-arg /var/lib/containers/ssd0/runroot --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/ssd0/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea COREDUMP_CWD /run/s6/legacy-services/tdarr_server COREDUMP_ENVIRON NVIDIA_VISIBLE_DEVICES=all nodeName=fedora-node ffmpegVersion=ffmpeg version 6.0-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers\nbuilt with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)\nconfiguration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc\nlibavutil 58. 2.100 / 58. 2.100\nlibavcodec 60. 3.100 / 60. 3.100\nlibavformat 60. 3.100 / 60. 3.100\nlibavdevice 60. 1.100 / 60. 1.100\nlibavfilter 9. 3.100 / 9. 3.100\nlibswscale 7. 1.100 / 7. 1.100\nlibswresample 4. 10.100 / 4. 10.100\nlibpostproc 57. 1.100 / 57. 1.100\n PUID=0 HOSTNAME=fedora S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 LANGUAGE=en_US.UTF-8 serverPort=8267 webUIPort=8264 UMASK=002 inContainer=true PWD=/run/s6/legacy-services/tdarr_server NVIDIA_DRIVER_CAPABILITIES=all TZ=America/New_York container=podman internalNode=true HOME=/home/Tdarr LANG=en_US.UTF-8 PGID=568 S6_VERBOSITY=1 S6_STAGE2_HOOK=/docker-mods BASE_LINK=https://tdarrs.s3.us-west-000.backblazeb2.com/dev/versions/2.17.01 NODE_PORT=8267 TERM=xterm WEB_UI_PORT=8265 UMASK_SET=000 SHLVL=0 LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri SERVER_PORT=8266 serverIP=0.0.0.0 PATH=/command:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HANDBRAKE=1.6.1 NODE_ENV=production tdarrConfig={\"serverPort\":\"8267\",\"webUIPort\":\"8264\",\"serverIP\":\"0.0.0.0\",\"handbrakePath\":\"\",\"ffmpegPath\":\"\",\"logLevel\":\"INFO\",\"mkvpropeditPath\":\"\",\"ccextractorPath\":\"\",\"openBrowser\":true,\"cronPluginUpdate\":\"\",\"platform_arch_isdocker\":\"linux_x64_docker_true\"} handbrakeVersion=[00:00:18] Compile-time hardening features are enabled\n[00:00:18] qsv: not available on this system\n[00:00:18] nvenc: version 12.1 is available\n[00:00:18] nvdec: is not compiled into this build\n[00:00:18] hb_init: starting libhb thread\n[00:00:18] thread 7f1f05400700 started (\"libhb\")\nHandBrake has exited.\nHandBrake 1.6.1\n\n mkvpropeditVersion= NODE_CHANNEL_FD=3 NODE_CHANNEL_SERIALIZATION_MODE=json COREDUMP_EXE /usr/bin/node COREDUMP_FILENAME /var/lib/systemd/coredump/core.Tdarr_Server.0.551005cd73fd4320ae00c3c7fd7518c0.187309.1709615091000000.zst COREDUMP_GID 568 COREDUMP_HOSTNAME fedora COREDUMP_OPEN_FDS [no data] COREDUMP_PACKAGE_JSON {"elfType":"coredump","elfArchitecture":"AMD x86-64","/usr/bin/node":{"type":"rpm","name":"nodejs18","version":"18.19.0-1.fc38","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:38","buildId":"7aebc7ddb910e7aea69b7e4551441526a3cd6f22"}} COREDUMP_PACKAGE_NAME nodejs18 COREDUMP_PACKAGE_VERSION 18.19.0-1.fc38 COREDUMP_PID 187309 COREDUMP_PROC_AUXV [binary data] COREDUMP_PROC_CGROUP 0::/machine.slice/libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope/container COREDUMP_PROC_LIMITS Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 4194304 4194304 processes Max open files 1048576 1048576 files Max locked memory 8388608 8388608 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127624 127624 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us COREDUMP_PROC_MAPS [no data] COREDUMP_PROC_MOUNTINFO 16461 1819 0:181 / / rw,relatime - overlay overlay rw,lowerdir=/var/lib/containers/ssd0/storage/overlay/l/LFDATJ7XQUKN5ZHJHUMXIFUVLN:/var/lib/containers/ssd0/storage/overlay/l/L2KWNU6VZXF3ZH5Q4BVEO6JSD4:/var/lib/containers/ssd0/storage/overlay/l/VOMBLOAOVRYHH3VYHJIKGEBKVR:/var/lib/containers/ssd0/storage/overlay/l/UKTIBT2LLJAIZVLTNJ7AJ5DT3B:/var/lib/containers/ssd0/storage/overlay/l/FX52UXG7LNPURPMAGPTQTQXKFO:/var/lib/containers/ssd0/storage/overlay/l/OLLMIWUI2V4ELDYWWX4G7QA3Z7:/var/lib/containers/ssd0/storage/overlay/l/2ZWVT47A6VBALPAMDWFZ4WILGO:/var/lib/containers/ssd0/storage/overlay/l/LSI2ZU2HQ2LDVBSAWBQJWERFLP:/var/lib/containers/ssd0/storage/overlay/l/63JLASNSXLZHSYGJCSJMIH5A3G,upperdir=/var/lib/containers/ssd0/storage/overlay/a1f8f9e2f073627f50b9d3a6f5a2ab7cd39bc3856410dd03ff3f7cce6f5fc387/diff,workdir=/var/lib/containers/ssd0/storage/overlay/a1f8f9e2f073627f50b9d3a6f5a2ab7cd39bc3856410dd03ff3f7cce6f5fc387/work 16462 16461 0:164 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64 16463 16461 0:42 / /media-pool rw,noatime - zfs media-pool rw,xattr,noacl,casesensitive 16464 16463 0:50 / /media-pool/rsync-folder rw,noatime - zfs media-pool/rsync-folder rw,xattr,noacl,casesensitive 16465 16463 0:52 / /media-pool/temp rw,noatime - zfs media-pool/temp rw,xattr,noacl,casesensitive 16466 16463 0:48 / /media-pool/downloads rw,noatime - zfs media-pool/downloads rw,xattr,noacl,casesensitive 16467 16466 0:61 / /media-pool/downloads/usenet-incomplete rw,noatime - zfs media-pool/downloads/usenet-incomplete rw,xattr,noacl,casesensitive 16468 16466 0:59 / /media-pool/downloads/nzb rw,noatime - zfs media-pool/downloads/nzb rw,xattr,noacl,casesensitive 16469 16466 0:57 / /media-pool/downloads/completed rw,noatime - zfs media-pool/downloads/completed rw,xattr,noacl,casesensitive 16470 16469 0:79 / /media-pool/downloads/completed/shows rw,noatime - zfs media-pool/downloads/completed/shows rw,xattr,noacl,casesensitive 16471 16469 0:78 / /media-pool/downloads/completed/movies rw,noatime - zfs media-pool/downloads/completed/movies rw,xattr,noacl,casesensitive 16472 16469 0:80 / /media-pool/downloads/completed/games rw,noatime - zfs media-pool/downloads/completed/games rw,xattr,noacl,casesensitive 16473 16469 0:83 / /media-pool/downloads/completed/xxx rw,noatime - zfs media-pool/downloads/completed/xxx rw,xattr,noacl,casesensitive 16474 16469 0:84 / /media-pool/downloads/completed/youtube-dl rw,noatime - zfs media-pool/downloads/completed/youtube-dl rw,xattr,noacl,casesensitive 16475 16469 0:82 / /media-pool/downloads/completed/torrents rw,noatime - zfs media-pool/downloads/completed/torrents rw,xattr,noacl,casesensitive 16476 16469 0:81 / /media-pool/downloads/completed/music rw,noatime - zfs media-pool/downloads/completed/music rw,xattr,noacl,casesensitive 16477 16466 0:63 / /media-pool/downloads/usenet-watch rw,noatime - zfs media-pool/downloads/usenet-watch rw,xattr,noacl,casesensitive 16478 16466 0:60 / /media-pool/downloads/torrent-blackhole rw,noatime - zfs media-pool/downloads/torrent-blackhole rw,xattr,noacl,casesensitive 16479 16466 0:58 / /media-pool/downloads/nzbackup rw,noatime - zfs media-pool/downloads/nzbackup rw,xattr,noacl,casesensitive 16480 16463 0:51 / /media-pool/nextcloud-storage rw,noatime - zfs media-pool/nextcloud-storage rw,xattr,noacl,casesensitive 16481 16480 0:65 / /media-pool/nextcloud-storage/backup rw,noatime - zfs media-pool/nextcloud-storage/backup rw,xattr,noacl,casesensitive 16482 16480 0:64 / /media-pool/nextcloud-storage/users-storage rw,noatime - zfs media-pool/nextcloud-storage/users-storage rw,xattr,noacl,casesensitive 16483 16480 0:62 / /media-pool/nextcloud-storage/ncdata rw,noatime - zfs media-pool/nextcloud-storage/ncdata rw,xattr,noacl,casesensitive 16484 16483 0:85 / /media-pool/nextcloud-storage/ncdata/dan rw,noatime - zfs media-pool/nextcloud-storage/ncdata/dan rw,xattr,noacl,casesensitive 16485 16463 0:47 / /media-pool/library rw,noatime - zfs media-pool/library rw,xattr,noacl,casesensitive COREDUMP_PROC_STATUS Name: Tdarr_Server Umask: 0002 State: S (sleeping) Tgid: 187309 Ngid: 0 Pid: 187309 PPid: 186914 TracerPid: 0 Uid: 0 0 0 0 Gid: 568 568 568 568 FDSize: 64 Groups: 0 39 100 568 NStgid: 187309 380 NSpid: 187309 380 NSpgid: 186914 233 NSsid: 186914 233 Kthread: 0 VmPeak: 6230296 kB VmSize: 6230044 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 4874872 kB VmRSS: 4747836 kB RssAnon: 4733572 kB RssFile: 14264 kB RssShmem: 0 kB VmData: 5467808 kB VmStk: 132 kB VmExe: 79268 kB VmLib: 4580 kB VmPTE: 127312 kB VmSwap: 660572 kB HugetlbPages: 0 kB CoreDumping: 1 THP_enabled: 1 untag_mask: 0xffffffffffffffff Threads: 11 SigQ: 3/127624 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000001001000 SigCgt: 0000000180004603 CapInh: 0000000000000000 CapPrm: 000001ffffffffff CapEff: 000001ffffffffff CapBnd: 000001ffffffffff CapAmb: 0000000000000000 NoNewPrivs: 0 Seccomp: 0 Seccomp_filters: 0 Speculation_Store_Bypass: thread vulnerable SpeculationIndirectBranch: conditional enabled Cpus_allowed: ff Cpus_allowed_list: 0-7 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 83758 nonvoluntary_ctxt_switches: 180134 x86_Thread_features: x86_Thread_features_locked: COREDUMP_RLIMIT 18446744073709551615 COREDUMP_ROOT / COREDUMP_SIGNAL 6 COREDUMP_SIGNAL_NAME SIGABRT COREDUMP_SLICE machine.slice COREDUMP_TIMESTAMP 1709615091000000 COREDUMP_UID 0 COREDUMP_UNIT libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope MESSAGE_ID fc2e22bc6ee647b6b90729ab34a250b1 PRIORITY 2 SYSLOG_IDENTIFIER systemd-coredump _BOOT_ID 551005cd73fd4320ae00c3c7fd7518c0 _CAP_EFFECTIVE 0 _CMDLINE /usr/lib/systemd/systemd-coredump _COMM systemd-coredum _EXE /usr/lib/systemd/systemd-coredump _GID 990 _HOSTNAME fedora _MACHINE_ID 089898983ea044f5b9c7a12a68c5de46 _PID 188780 _RUNTIME_SCOPE system _SOURCE_REALTIME_TIMESTAMP 1709615304078200 _SYSTEMD_CGROUP /system.slice/system-systemd\x2dcoredump.slice/systemd-coredump@0-188772-0.service _SYSTEMD_INVOCATION_ID 8ebf39242c004095a5fe0a4caf615d0e _SYSTEMD_SLICE system-systemd\x2dcoredump.slice _SYSTEMD_UNIT systemd-coredump@0-188772-0.service _TRANSPORT journal _UID 990 __CURSOR s=ce96d9095da74b739a4489146aa9dc11;i=e6e2830;b=551005cd73fd4320ae00c3c7fd7518c0;m=1b4d84d7f;t=612e2d4e6f387;x=bf4ac59fda0efcf6 __MONOTONIC_TIMESTAMP 7329041791 __REALTIME_TIMESTAMP 1709615304078215

5:31 PM Process 221685 (Tdarr_Server) of user 0 dumped core. Module /usr/bin/node from rpm nodejs18-18.19.0-1.fc38.x86_64 Stack trace of thread 408: #0 0x00007f2c0b3ad00b n/a (/usr/lib/x86_64-linux-gnu/libc-2.31.so + 0x4300b) #1 0x0000000000a94834 systemd-coredump

systemd-coredump Process 221685 (Tdarr_Server) of user 0 dumped core. Module /usr/bin/node from rpm nodejs18-18.19.0-1.fc38.x86_64 Stack trace of thread 408: #0 0x00007f2c0b3ad00b n/a (/usr/lib/x86_64-linux-gnu/libc-2.31.so + 0x4300b) #1 0x0000000000a94834 n/a (/usr/bin/node + 0x694834) #2 0x0000000000d647c0 n/a (/usr/bin/node + 0x9647c0) #3 0x0000000000d64b67 n/a (/usr/bin/node + 0x964b67) #4 0x0000000000f42265 n/a (/usr/bin/node + 0xb42265) #5 0x0000000000f5474d n/a (/usr/bin/node + 0xb5474d) #6 0x0000000000f2ee4e n/a (/usr/bin/node + 0xb2ee4e) #7 0x0000000000f30217 n/a (/usr/bin/node + 0xb30217) #8 0x0000000000f10760 n/a (/usr/bin/node + 0xb10760) #9 0x0000000000f081d4 n/a (/usr/bin/node + 0xb081d4) #10 0x0000000000f0a488 n/a (/usr/bin/node + 0xb0a488) #11 0x00000000011e7add n/a (/usr/bin/node + 0xde7add) #12 0x00000000012f6ddd n/a (/usr/bin/node + 0xef6ddd) #13 0x00000000017035b9 n/a (/usr/bin/node + 0x13035b9) #14 0x0000000005e2d550 n/a (n/a + 0x0) #15 0x0000000005ef79c2 n/a (n/a + 0x0) #16 0x0000000005efa850 n/a (n/a + 0x0) #17 0x00000000016bb2ff n/a (/usr/bin/node + 0x12bb2ff) #18 0x0000000001753bb1 n/a (/usr/bin/node + 0x1353bb1) #19 0x00000000016aca7b n/a (/usr/bin/node + 0x12aca7b) #20 0x0000000001685b43 n/a (/usr/bin/node + 0x1285b43) #21 0x0000000000ea548a n/a (/usr/bin/node + 0xaa548a) #22 0x0000000000ea6090 n/a (/usr/bin/node + 0xaa6090) #23 0x0000000000ea679a n/a (/usr/bin/node + 0xaa679a) #24 0x0000000000ed3c9a n/a (/usr/bin/node + 0xad3c9a) #25 0x0000000000ed4072 n/a (/usr/bin/node + 0xad4072) #26 0x0000000000ac4c12 n/a (/usr/bin/node + 0x6c4c12) #27 0x0000000000ac50f6 n/a (/usr/bin/node + 0x6c50f6) #28 0x0000000000ad722d n/a (/usr/bin/node + 0x6d722d) #29 0x0000000000b88c32 n/a (/usr/bin/node + 0x788c32) #30 0x0000000000b8c6d1 n/a (/usr/bin/node + 0x78c6d1) #31 0x0000000000b80974 n/a (/usr/bin/node + 0x780974) #32 0x000000000166224d n/a (/usr/bin/node + 0x126224d) #33 0x0000000001666a36 n/a (/usr/bin/node + 0x1266a36) #34 0x0000000001678f74 n/a (/usr/bin/node + 0x1278f74) #35 0x000000000166739e n/a (/usr/bin/node + 0x126739e) #36 0x0000000000ac5a6d n/a (/usr/bin/node + 0x6c5a6d) #37 0x0000000000bc8d14 n/a (/usr/bin/node + 0x7c8d14) #38 0x0000000000b3ec18 n/a (/usr/bin/node + 0x73ec18) #39 0x0000000000b4272f n/a (/usr/bin/node + 0x74272f) #40 0x00007f2c0b38e083 n/a (/usr/lib/x86_64-linux-gnu/libc-2.31.so + 0x24083) ELF object binary architecture: AMD x86-64 CODE_FILE ../src/coredump/coredump.c CODE_FUNC submit_coredump CODE_LINE 1053 COREDUMP_CGROUP /machine.slice/libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope/container COREDUMP_CMDLINE Tdarr_Server COREDUMP_COMM Tdarr_Server COREDUMP_CONTAINER_CMDLINE /usr/bin/conmon --api-version 1 -c d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea -u d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea -r /usr/bin/crun -b /var/lib/containers/ssd0/storage/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata -p /var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/pidfile -n tdarr --exit-dir /run/libpod/exits --full-attach -s -l journald --log-level warning --syslog --log-size-max 10000000 --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/oci-log --conmon-pidfile /var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/ssd0/storage --exit-command-arg --runroot --exit-command-arg /var/lib/containers/ssd0/runroot --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/ssd0/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea COREDUMP_CWD /run/s6/legacy-services/tdarr_server COREDUMP_ENVIRON NVIDIA_VISIBLE_DEVICES=all nodeName=fedora-node ffmpegVersion=ffmpeg version 6.0-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers\nbuilt with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)\nconfiguration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc\nlibavutil 58. 2.100 / 58. 2.100\nlibavcodec 60. 3.100 / 60. 3.100\nlibavformat 60. 3.100 / 60. 3.100\nlibavdevice 60. 1.100 / 60. 1.100\nlibavfilter 9. 3.100 / 9. 3.100\nlibswscale 7. 1.100 / 7. 1.100\nlibswresample 4. 10.100 / 4. 10.100\nlibpostproc 57. 1.100 / 57. 1.100\n PUID=0 HOSTNAME=fedora S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 LANGUAGE=en_US.UTF-8 serverPort=8267 webUIPort=8264 UMASK=002 inContainer=true PWD=/run/s6/legacy-services/tdarr_server NVIDIA_DRIVER_CAPABILITIES=all TZ=America/New_York container=podman internalNode=true HOME=/home/Tdarr LANG=en_US.UTF-8 PGID=568 S6_VERBOSITY=1 S6_STAGE2_HOOK=/docker-mods BASE_LINK=https://tdarrs.s3.us-west-000.backblazeb2.com/dev/versions/2.17.01 NODE_PORT=8267 TERM=xterm WEB_UI_PORT=8265 UMASK_SET=000 SHLVL=0 LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri SERVER_PORT=8266 serverIP=0.0.0.0 PATH=/command:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HANDBRAKE=1.6.1 NODE_ENV=production tdarrConfig={\"serverPort\":\"8267\",\"webUIPort\":\"8264\",\"serverIP\":\"0.0.0.0\",\"handbrakePath\":\"\",\"ffmpegPath\":\"\",\"logLevel\":\"INFO\",\"mkvpropeditPath\":\"\",\"ccextractorPath\":\"\",\"openBrowser\":true,\"cronPluginUpdate\":\"\",\"platform_arch_isdocker\":\"linux_x64_docker_true\"} handbrakeVersion=[17:19:12] Compile-time hardening features are enabled\n[17:19:13] qsv: not available on this system\n[17:19:13] nvenc: version 12.1 is available\n[17:19:13] nvdec: is not compiled into this build\n[17:19:13] hb_init: starting libhb thread\n[17:19:13] thread 7fdb02000700 started (\"libhb\")\nHandBrake has exited.\nHandBrake 1.6.1\n\n mkvpropeditVersion= NODE_CHANNEL_FD=3 NODE_CHANNEL_SERIALIZATION_MODE=json COREDUMP_EXE /usr/bin/node COREDUMP_FILENAME /var/lib/systemd/coredump/core.Tdarr_Server.0.11e540d69c4a465fa5e01c346f0d6f5f.221685.1709591313000000.zst COREDUMP_GID 568 COREDUMP_HOSTNAME fedora COREDUMP_OPEN_FDS 0:/dev/null pos: 0 flags: 02100000 mnt_id: 33394 ino: 130 1:pipe:[406285] pos: 0 flags: 02000001 mnt_id: 15 ino: 406285 2:pipe:[406286] pos: 0 flags: 02000001 mnt_id: 15 ino: 406286 3:socket:[417873] pos: 0 flags: 02004002 mnt_id: 9 ino: 417873 scm_fds: 0 4:anon_inode:[eventpoll] pos: 0 flags: 02000002 mnt_id: 16 ino: 84 5:pipe:[417876] pos: 0 flags: 02000000 mnt_id: 15 ino: 417876 6:pipe:[417876] pos: 0 flags: 02000001 mnt_id: 15 ino: 417876 7:pipe:[417877] pos: 0 flags: 02004000 mnt_id: 15 ino: 417877 8:pipe:[417877] pos: 0 flags: 02004001 mnt_id: 15 ino: 417877 9:anon_inode:[eventfd] pos: 0 flags: 02004002 mnt_id: 16 ino: 84 eventfd-count: 0 eventfd-id: 216 eventfd-semaphore: 0 10:anon_inode:[eventpoll] pos: 0 flags: 02000002 mnt_id: 16 ino: 84 tfd: 11 events: 19 data: b pos:0 ino:65a87 sdev:e tfd: 13 events: 19 data: d pos:0 ino:54 sdev:f 11:pipe:[416391] pos: 0 flags: 02004000 mnt_id: 15 ino: 416391 12:pipe:[416391] pos: 0 flags: 02004001 mnt_id: 15 ino: 416391 13:anon_inode:[eventfd] pos: 0 flags: 02004002 mnt_id: 16 ino: 84 eventfd-count: 0 eventfd-id: 217 eventfd-semaphore: 0 14:anon_inode:[eventpoll] pos: 0 flags: 02000002 mnt_id: 16 ino: 84 tfd: 15 events: 19 data: f pos:0 ino:66056 sdev:e tfd: 3 events: 19 data: 3 pos:0 ino:66051 sdev:8 tfd: 17 events: 19 data: 11 pos:0 ino:54 sdev:f 15:pipe:[417878] pos: 0 flags: 02004000 mnt_id: 15 ino: 417878 16:pipe:[417878] pos: 0 flags: 02004001 mnt_id: 15 ino: 417878 17:anon_inode:[eventfd] pos: 0 flags: 02004002 mnt_id: 16 ino: 84 eventfd-count: 1 eventfd-id: 218 eventfd-semaphore: 0 18:/dev/null pos: 0 flags: 02100000 mnt_id: 33394 ino: 130 19:/app/logs/Tdarr_Server_Log.txt pos: 849759 flags: 02102001 mnt_id: 35050 ino: 33987 21:/dev/urandom pos: 0 flags: 0100000 mnt_id: 33394 ino: 319 COREDUMP_PACKAGE_JSON {"elfType":"coredump","elfArchitecture":"AMD x86-64","/usr/bin/node":{"type":"rpm","name":"nodejs18","version":"18.19.0-1.fc38","architecture":"x86_64","osCpe":"cpe:/o:fedoraproject:fedora:38","buildId":"7aebc7ddb910e7aea69b7e4551441526a3cd6f22"}} COREDUMP_PACKAGE_NAME nodejs18 COREDUMP_PACKAGE_VERSION 18.19.0-1.fc38 COREDUMP_PID 221685 COREDUMP_PROC_AUXV [binary data] COREDUMP_PROC_CGROUP 0::/machine.slice/libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope/container COREDUMP_PROC_LIMITS Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 4194304 4194304 processes Max open files 1048576 1048576 files Max locked memory 8388608 8388608 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127618 127618 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us COREDUMP_PROC_MAPS 00400000-05169000 r-xp 00000000 e6:10 899976 /usr/bin/node 05369000-0536c000 r--p 04d69000 e6:10 899976 /usr/bin/node 0536c000-05387000 rw-p 04d6c000 e6:10 899976 /usr/bin/node 05387000-053aa000 rw-p 00000000 00:00 0 05985000-05d7a000 rw-p 00000000 00:00 0 [heap] 05d7a000-05d80000 ---p 00000000 00:00 0 05d80000-05d82000 rw-p 00000000 00:00 0 05d82000-05d83000 ---p 00000000 00:00 0 05d83000-05db2000 r-xp 00000000 00:00 0 05db2000-05dbf000 ---p 00000000 00:00 0 05dbf000-05dc0000 ---p 00000000 00:00 0 05dc0000-05dc2000 rw-p 00000000 00:00 0 05dc2000-05dc3000 ---p 00000000 00:00 0 05dc3000-05dff000 r-xp 00000000 00:00 0 05dff000-05e00000 ---p 00000000 00:00 0 05e00000-05e02000 rw-p 00000000 00:00 0 05e02000-05e03000 ---p 00000000 00:00 0 05e03000-05e3f000 r-xp 00000000 00:00 0 05e3f000-05e40000 ---p 00000000 00:00 0 05e40000-05e42000 ---p 00000000 00:00 0 05e42000-05e43000 ---p 00000000 00:00 0 05e43000-05e7f000 ---p 00000000 00:00 0 05e7f000-05e80000 ---p 00000000 00:00 0 05e80000-05e82000 ---p 00000000 00:00 0 05e82000-05e83000 ---p 00000000 00:00 0 05e83000-05ebf000 ---p 00000000 00:00 0 05ebf000-05ec0000 ---p 00000000 00:00 0 05ec0000-05ec2000 rw-p 00000000 00:00 0 05ec2000-05ec3000 ---p 00000000 00:00 0 05ec3000-05eff000 r-xp 00000000 00:00 0 05eff000-05f00000 ---p 00000000 00:00 0 05f00000-05f02000 ---p 00000000 00:00 0 05f02000-05f03000 ---p 00000000 00:00 0 05f03000-05f3f000 ---p 00000000 00:00 0 05f3f000-0dd7a000 ---p 00000000 00:00 0 114180000-1141c0000 rw-p 00000000 00:00 0 1cca00000-1cca40000 rw-p 00000000 00:00 0 2f1640000-2f1680000 rw-p 00000000 00:00 0 3faf00000-3faf40000 rw-p 00000000 00:00 0 48fc80000-48fcc0000 rw-p 00000000 00:00 0 650840000-650880000 rw-p 00000000 00:00 0 69cc80000-69ccc0000 rw-p 00000000 00:00 0 758b00000-758b40000 rw-p 00000000 00:00 0 75d900000-75d940000 rw-p 00000000 00:00 0 77e080000-77e0c0000 rw-p 00000000 00:00 0 8bb040000-8bb080000 rw-p 00000000 00:00 0 a89a80000-a89ac0000 rw-p 00000000 00:00 0 c0eb40000-c0eb80000 rw-p 00000000 00:00 0 c69c40000-c69c80000 rw-p 00000000 00:00 0 e82400000-e82440000 rw-p 00000000 00:00 0 f3c800000-f3c840000 rw-p 00000000 00:00 0 fd9a00000-fd9a40000 rw-p 00000000 00:00 0 1002a80000-1002ac0000 rw-p 00000000 00:00 0 10459c0000-1045a00000 rw-p 00000000 00:00 0 10801c0000-1080200000 rw-p 00000000 00:00 0 1101080000-11010c0000 rw-p 00000000 00:00 0 12baa40000-12baa80000 rw-p 00000000 00:00 0 12da840000-12da880000 ---p 00000000 00:00 0 1325340000-1325380000 rw-p 00000000 00:00 0 137f5c0000-137f600000 rw-p 00000000 00:00 0 139cdc0000-139ce00000 rw-p 00000000 00:00 0 1529d80000-1529dc0000 rw-p 00000000 00:00 0 179b4c0000-179b500000 rw-p 00000000 00:00 0 18c8180000-18c81c0000 rw-p 00000000 00:00 0 1940c00000-1940c40000 rw-p 00000000 00:00 0 19dda40000-19dda80000 rw-p 00000000 00:00 0 1b17f40000-1b17f80000 rw-p 00000000 00:00 0 1b35a40000-1b35a80000 ---p 00000000 00:00 0 1c62cc0000-1c62d00000 rw-p 00000000 00:00 0 1cb9800000-1cb9840000 rw-p 00000000 00:00 0 1e1ed40000-1e1ed80000 rw-p 00000000 00:00 0 1e29b40000-1e29b80000 rw-p 00000000 00:00 0 1ef9a00000-1ef9a40000 rw-p 00000000 00:00 0 1f016c0000-1f01700000 rw-p 00000000 00:00 0 1fcc1c0000-1fcc200000 rw-p 00000000 00:00 0 1ffdcc0000-1ffdd00000 rw-p 00000000 00:00 0 201ec80000-201ecc0000 rw-p 00000000 00:00 0 209f640000-209f680000 rw-p 00000000 00:00 0 21d89c0000-21d8a00000 rw-p 00000000 00:00 0 2211b00000-2211b40000 rw-p 00000000 00:00 0 227f500000-227f540000 rw-p 00000000 00:00 0 24d9a40000-24d9a80000 rw-p 00000000 00:00 0 2608680000-26086c0000 rw-p 00000000 00:00 0 269e340000-269e380000 rw-p 00000000 00:00 0 26af200000-26af240000 rw-p 00000000 00:00 0 277a000000-277a040000 rw-p 00000000 00:00 0 2907600000-2907640000 rw-p 00000000 00:00 0 296ac00000-296ac40000 rw-p 00000000 00:00 0 29847c0000-2984800000 rw-p 00000000 00:00 0 2ab3100000-2ab3140000 rw-p 00000000 00:00 0 COREDUMP_PROC_MOUNTINFO 33393 1583 0:176 / / rw,relatime - overlay overlay rw,lowerdir=/var/lib/containers/ssd0/storage/overlay/l/LFDATJ7XQUKN5ZHJHUMXIFUVLN:/var/lib/containers/ssd0/storage/overlay/l/L2KWNU6VZXF3ZH5Q4BVEO6JSD4:/var/lib/containers/ssd0/storage/overlay/l/VOMBLOAOVRYHH3VYHJIKGEBKVR:/var/lib/containers/ssd0/storage/overlay/l/UKTIBT2LLJAIZVLTNJ7AJ5DT3B:/var/lib/containers/ssd0/storage/overlay/l/FX52UXG7LNPURPMAGPTQTQXKFO:/var/lib/containers/ssd0/storage/overlay/l/OLLMIWUI2V4ELDYWWX4G7QA3Z7:/var/lib/containers/ssd0/storage/overlay/l/2ZWVT47A6VBALPAMDWFZ4WILGO:/var/lib/containers/ssd0/storage/overlay/l/LSI2ZU2HQ2LDVBSAWBQJWERFLP:/var/lib/containers/ssd0/storage/overlay/l/63JLASNSXLZHSYGJCSJMIH5A3G,upperdir=/var/lib/containers/ssd0/storage/overlay/a1f8f9e2f073627f50b9d3a6f5a2ab7cd39bc3856410dd03ff3f7cce6f5fc387/diff,workdir=/var/lib/containers/ssd0/storage/overlay/a1f8f9e2f073627f50b9d3a6f5a2ab7cd39bc3856410dd03ff3f7cce6f5fc387/work,uuid=on 33394 33393 0:180 / /dev rw,nosuid - tmpfs tmpfs rw,size=65536k,mode=755,inode64 33395 33393 0:43 / /media-pool rw,noatime - zfs media-pool rw,xattr,noacl,casesensitive 33396 33395 0:51 / /media-pool/temp rw,noatime - zfs media-pool/temp rw,xattr,noacl,casesensitive 33397 33395 0:52 / /media-pool/rsync-folder rw,noatime - zfs media-pool/rsync-folder rw,xattr,noacl,casesensitive 33398 33395 0:49 / /media-pool/library rw,noatime - zfs media-pool/library rw,xattr,noacl,casesensitive 33399 33398 0:56 / /media-pool/library/shows rw,noatime - zfs media-pool/library/shows rw,xattr,noacl,casesensitive 33400 33398 0:67 / /media-pool/library/youtubedl rw,noatime - zfs media-pool/library/youtubedl rw,xattr,noacl,casesensitive 33401 33400 0:1820 / /media-pool/library/youtubedl/.zfs/snapshot/2024-03-03-070000 rw,relatime - zfs media-pool/library/youtubedl@2024-03-03-070000 ro,xattr,noacl,casesensitive 33402 33398 0:66 / /media-pool/library/movies rw,noatime - zfs media-pool/library/movies rw,xattr,noacl,casesensitive 33403 33402 0:200 / /media-pool/library/movies/.zfs/snapshot/2024-02-26-010000 rw,relatime - zfs media-pool/library/movies@2024-02-26-010000 ro,xattr,noacl,casesensitive 33404 33402 0:202 / /media-pool/library/movies/.zfs/snapshot/2024-02-24-220000 rw,relatime - zfs media-pool/library/movies@2024-02-24-220000 ro,xattr,noacl,casesensitive 33405 33402 0:204 / /media-pool/library/movies/.zfs/snapshot/2024-02-03-000000 rw,relatime - zfs media-pool/library/movies@2024-02-03-000000 ro,xattr,noacl,casesensitive 33406 33402 0:206 / /media-pool/library/movies/.zfs/snapshot/2024-03-03-090000 rw,relatime - zfs media-pool/library/movies@2024-03-03-090000 ro,xattr,noacl,casesensitive 33407 33402 0:208 / /media-pool/library/movies/.zfs/snapshot/2024-03-04-020000 rw,relatime - zfs media-pool/library/movies@2024-03-04-020000 ro,xattr,noacl,casesensitive 33408 33402 0:210 / /media-pool/library/movies/.zfs/snapshot/2024-02-27-130000 rw,relatime - zfs media-pool/library/movies@2024-02-27-130000 ro,xattr,noacl,casesensitive 33409 33402 0:212 / /media-pool/library/movies/.zfs/snapshot/2024-03-04-050000 rw,relatime - zfs media-pool/library/movies@2024-03-04-050000 ro,xattr,noacl,casesensitive 33410 33402 0:214 / /media-pool/library/movies/.zfs/snapshot/2024-02-27-140000 rw,relatime - zfs media-pool/library/movies@2024-02-27-140000 ro,xattr,noacl,casesensitive 33411 33402 0:216 / /media-pool/library/movies/.zfs/snapshot/2024-02-26-060000 rw,relatime - zfs media-pool/library/movies@2024-02-26-060000 ro,xattr,noacl,casesensitive 33412 33402 0:218 / /media-pool/library/movies/.zfs/snapshot/2024-02-18-000000 rw,relatime - zfs media-pool/library/movies@2024-02-18-000000 ro,xattr,noacl,casesensitive 33413 33402 0:220 / /media-pool/library/movies/.zfs/snapshot/2024-03-01-230000 rw,relatime - zfs media-pool/library/movies@2024-03-01-230000 ro,xattr,noacl,casesensitive 33414 33402 0:222 / /media-pool/library/movies/.zfs/snapshot/2024-03-03-000000 rw,relatime - zfs media-pool/library/movies@2024-03-03-000000 ro,xattr,noacl,casesensitive COREDUMP_PROC_STATUS Name: Tdarr_Server Umask: 0002 State: S (sleeping) Tgid: 221685 Ngid: 0 Pid: 221685 PPid: 220937 TracerPid: 0 Uid: 0 0 0 0 Gid: 568 568 568 568 FDSize: 64 Groups: 0 39 100 568 NStgid: 221685 408 NSpid: 221685 408 NSpgid: 220937 253 NSsid: 220937 253 Kthread: 0 VmPeak: 6308376 kB VmSize: 6307868 kB VmLck: 0 kB VmPin: 0 kB VmHWM: 5039620 kB VmRSS: 4486932 kB RssAnon: 4473296 kB RssFile: 13636 kB RssShmem: 0 kB VmData: 5517256 kB VmStk: 132 kB VmExe: 79268 kB VmLib: 4340 kB VmPTE: 128300 kB VmSwap: 969132 kB HugetlbPages: 0 kB CoreDumping: 1 THP_enabled: 1 untag_mask: 0xffffffffffffffff Threads: 11 SigQ: 3/127618 SigPnd: 0000000000000000 ShdPnd: 0000000000000000 SigBlk: 0000000000000000 SigIgn: 0000000001001000 SigCgt: 0000000180004603 CapInh: 0000000000000000 CapPrm: 000001ffffffffff CapEff: 000001ffffffffff CapBnd: 000001ffffffffff CapAmb: 0000000000000000 NoNewPrivs: 0 Seccomp: 0 Seccomp_filters: 0 Speculation_Store_Bypass: thread vulnerable SpeculationIndirectBranch: conditional enabled Cpus_allowed: ff Cpus_allowed_list: 0-7 Mems_allowed: 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000001 Mems_allowed_list: 0 voluntary_ctxt_switches: 150654 nonvoluntary_ctxt_switches: 223091 x86_Thread_features: x86_Thread_features_locked: COREDUMP_RLIMIT 18446744073709551615 COREDUMP_ROOT / COREDUMP_SIGNAL 6 COREDUMP_SIGNAL_NAME SIGABRT COREDUMP_SLICE machine.slice COREDUMP_TIMESTAMP 1709591313000000 COREDUMP_UID 0 COREDUMP_UNIT libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope MESSAGE_ID fc2e22bc6ee647b6b90729ab34a250b1 PRIORITY 2 SYSLOG_IDENTIFIER systemd-coredump _BOOT_ID 11e540d69c4a465fa5e01c346f0d6f5f _CAP_EFFECTIVE 0 _CMDLINE /usr/lib/systemd/systemd-coredump _COMM systemd-coredum _EXE /usr/lib/systemd/systemd-coredump _GID 990 _HOSTNAME fedora _MACHINE_ID 089898983ea044f5b9c7a12a68c5de46 _PID 227250 _RUNTIME_SCOPE system _SOURCE_REALTIME_TIMESTAMP 1709591483382114 _SYSTEMD_CGROUP /system.slice/system-systemd\x2dcoredump.slice/systemd-coredump@1-227249-0.service _SYSTEMD_INVOCATION_ID fd87175c7161470caf8370549fd29a35 _SYSTEMD_SLICE system-systemd\x2dcoredump.slice _SYSTEMD_UNIT systemd-coredump@1-227249-0.service _TRANSPORT journal _UID 990 __CURSOR s=ce96d9095da74b739a4489146aa9dc11;i=e622b29;b=11e540d69c4a465fa5e01c346f0d6f5f;m=11fdf0d23;t=612dd4913f9ba;x=cee1c5df07657cde __MONOTONIC_TIMESTAMP 4829678883 __REALTIME_TIMESTAMP 1709591483382202

5:27 PM Process 222222 (tdarr-ffmpeg) of user 0 dumped core. Stack trace of thread 848: #0 0x00007f62c542d3f9 n/a (libnvidia-encode.so.1 + 0x2d3f9) #1 0x00007f630782f6f9 n/a (/usr/lib/jellyfin-ffmpeg/lib/libavcodec.so.60 + 0x5d26f9) #2 0x000056016433a0d0 n/a (n/a + 0x0) ELF object binary architecture: AMD x86-64 systemd-coredump

systemd-coredump Process 222222 (tdarr-ffmpeg) of user 0 dumped core. Stack trace of thread 848: #0 0x00007f62c542d3f9 n/a (libnvidia-encode.so.1 + 0x2d3f9) #1 0x00007f630782f6f9 n/a (/usr/lib/jellyfin-ffmpeg/lib/libavcodec.so.60 + 0x5d26f9) #2 0x000056016433a0d0 n/a (n/a + 0x0) ELF object binary architecture: AMD x86-64 CODE_FILE ../src/coredump/coredump.c CODE_FUNC submit_coredump CODE_LINE 1053 COREDUMP_CGROUP /machine.slice/libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope/container COREDUMP_CMDLINE tdarr-ffmpeg -c:v h264_cuvid -i $'/media-pool/library/shows/Roadkill (2012)/Roadkill.Season 10/Roadkill - S10E09 - Trash Metropolitan Road Trip! [WEBDL-1080p][8bit][h264][AAC 2.0]-BTN.mkv' -map 0 -c:v hevc_nvenc -profile:v main10 -pix_fmt:v p010le -qmin 0 -cq:v 24 -b:v 27989k -maxrate:v 28389k -preset slow -rc-lookahead 32 -spatial_aq:v 1 -aq-strength:v 15 -metadata:s:v:0 COPYRIGHT=processed -c:a copy -c:s copy -max_muxing_queue_size 9999 -analyzeduration 2147483647 -probesize 2147483647 $'/temp/tdarr-workDir-node-8INoznu0I-worker-bogus-bear-ts-1709590774555/1709590801782/Roadkill - S10E09 - Trash Metropolitan Road Trip! [WEBDL-1080p][8bit][h264][AAC 2.0]-BTN.mkv' COREDUMP_COMM tdarr-ffmpeg COREDUMP_CONTAINER_CMDLINE /usr/bin/conmon --api-version 1 -c d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea -u d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea -r /usr/bin/crun -b /var/lib/containers/ssd0/storage/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata -p /var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/pidfile -n tdarr --exit-dir /run/libpod/exits --full-attach -s -l journald --log-level warning --syslog --log-size-max 10000000 --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/oci-log --conmon-pidfile /var/lib/containers/ssd0/runroot/overlay-containers/d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/ssd0/storage --exit-command-arg --runroot --exit-command-arg /var/lib/containers/ssd0/runroot --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/ssd0/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea COREDUMP_CWD /app/Tdarr_Node COREDUMP_ENVIRON NVIDIA_VISIBLE_DEVICES=all nodeName=fedora-node ffmpegVersion=ffmpeg version 6.0-Jellyfin Copyright (c) 2000-2023 the FFmpeg developers\nbuilt with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.2)\nconfiguration: --prefix=/usr/lib/jellyfin-ffmpeg --target-os=linux --extra-version=Jellyfin --disable-doc --disable-ffplay --disable-ptx-compression --disable-static --disable-libxcb --disable-sdl2 --disable-xlib --enable-lto --enable-gpl --enable-version3 --enable-shared --enable-gmp --enable-gnutls --enable-chromaprint --enable-libdrm --enable-libass --enable-libfreetype --enable-libfribidi --enable-libfontconfig --enable-libbluray --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libopenmpt --enable-libdav1d --enable-libwebp --enable-libvpx --enable-libx264 --enable-libx265 --enable-libzvbi --enable-libzimg --enable-libfdk-aac --arch=amd64 --enable-libsvtav1 --enable-libshaderc --enable-libplacebo --enable-vulkan --enable-opencl --enable-vaapi --enable-amf --enable-libvpl --enable-ffnvcodec --enable-cuda --enable-cuda-llvm --enable-cuvid --enable-nvdec --enable-nvenc\nlibavutil 58. 2.100 / 58. 2.100\nlibavcodec 60. 3.100 / 60. 3.100\nlibavformat 60. 3.100 / 60. 3.100\nlibavdevice 60. 1.100 / 60. 1.100\nlibavfilter 9. 3.100 / 9. 3.100\nlibswscale 7. 1.100 / 7. 1.100\nlibswresample 4. 10.100 / 4. 10.100\nlibpostproc 57. 1.100 / 57. 1.100\n PUID=0 HOSTNAME=fedora S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0 LANGUAGE=en_US.UTF-8 serverPort=8267 webUIPort=8264 UMASK=002 inContainer=true PWD=/run/s6/legacy-services/tdarr_node NVIDIA_DRIVER_CAPABILITIES=all TZ=America/New_York container=podman internalNode=true HOME=/home/Tdarr LANG=en_US.UTF-8 PGID=568 S6_VERBOSITY=1 S6_STAGE2_HOOK=/docker-mods BASE_LINK=https://tdarrs.s3.us-west-000.backblazeb2.com/dev/versions/2.17.01 NODE_PORT=8267 TERM=xterm WEB_UI_PORT=8265 UMASK_SET=000 SHLVL=0 LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu LIBVA_DRIVERS_PATH=/usr/lib/x86_64-linux-gnu/dri SERVER_PORT=8266 serverIP=0.0.0.0 PATH=/command:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HANDBRAKE=1.6.1 NODE_ENV=production tdarrConfig={\"nodeName\":\"fedora-node\",\"serverIP\":\"0.0.0.0\",\"serverPort\":\"8267\",\"handbrakePath\":\"\",\"ffmpegPath\":\"\",\"mkvpropeditPath\":\"\",\"pathTranslators\":[{\"server\":\"\",\"node\":\"\"}],\"logLevel\":\"INFO\",\"priority\":-1,\"platform_arch_isdocker\":\"linux_x64_docker_true\",\"processPid\":252,\"cronPluginUpdate\":\"\",\"nodeID\":\"8INoznu0I\"} handbrakeVersion=[17:19:10] Compile-time hardening features are enabled\n[17:19:10] qsv: not available on this system\n[17:19:10] nvenc: version 12.1 is available\n[17:19:10] nvdec: is not compiled into this build\n[17:19:11] hb_init: starting libhb thread\n[17:19:11] thread 7f170e400700 started (\"libhb\")\nHandBrake has exited.\nHandBrake 1.6.1\n\n mkvpropeditVersion= COREDUMP_EXE /usr/lib/jellyfin-ffmpeg/ffmpeg COREDUMP_FILENAME /var/lib/systemd/coredump/core.tdarr-ffmpeg.0.11e540d69c4a465fa5e01c346f0d6f5f.222222.1709591207000000.zst COREDUMP_GID 568 COREDUMP_HOSTNAME fedora COREDUMP_OPEN_FDS [no data] COREDUMP_PACKAGE_JSON {"elfType":"coredump","elfArchitecture":"AMD x86-64"} COREDUMP_PID 222222 COREDUMP_PROC_AUXV [binary data] COREDUMP_PROC_CGROUP 0::/machine.slice/libpod-d1746344d0af7508e2564cd0f7b1422249470dda50f07c5b14bc870c091c04ea.scope/container COREDUMP_PROC_LIMITS Limit Soft Limit Hard Limit Units Max cpu time unlimited unlimited seconds Max file size unlimited unlimited bytes Max data size unlimited unlimited bytes Max stack size 8388608 unlimited bytes Max core file size unlimited unlimited bytes Max resident set unlimited unlimited bytes Max processes 4194304 4194304 processes Max open files 1048576 1048576 files Max locked memory 8388608 8388608 bytes Max address space unlimited unlimited bytes Max file locks unlimited unlimited locks Max pending signals 127618 127618 signals Max msgqueue size 819200 819200 bytes Max nice priority 0 0 Max realtime priority 0 0 Max realtime timeout unlimited unlimited us COREDUMP_PROC_MAPS

Reboot 3:11 PM Process 1383705 (systemd-timedat) of user 0 dumped core. Module libpcre2-8.so.0 from rpm pcre2-10.42-1.fc38.1.x86_64 systemd-coredump

12:17 PM Process 1333531 (ccextractor) of user 0 dumped core. Stack trace of thread 29278: #0 0x00007f17f24c500b n/a (n/a + 0x0) ELF object binary architecture: AMD x86-64 systemd-coredump

systemd-coredump Process 1333531 (ccextractor) of user 0 dumped core. Stack trace of thread 29278: #0 0x00007f17f24c500b n/a (n/a + 0x0) ELF object binary architecture: AMD x86-64 CODE_FILE ../src/coredump/coredump.c CODE_FUNC submit_coredump CODE_LINE 1053 COREDUMP_CGROUP /machine.slice/libpod-c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8.scope/container COREDUMP_CMDLINE ccextractor -debug -stdout -endat 01:00 $'/media-pool/library/movies/The Mask (1994)/Mask, The (1994) [imdb-tt0110475] - [Multi-French][Bluray-1080p][8bit][x264][DTS 5.1]-UKDHD.mkv' COREDUMP_COMM ccextractor COREDUMP_CONTAINER_CMDLINE /usr/bin/conmon --api-version 1 -c c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8 -u c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8 -r /usr/bin/crun -b /var/lib/containers/ssd0/storage/overlay-containers/c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8/userdata -p /var/lib/containers/ssd0/runroot/overlay-containers/c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8/userdata/pidfile -n tdarr --exit-dir /run/libpod/exits --full-attach -s -l journald --log-level warning --syslog --log-size-max 10000000 --runtime-arg --log-format=json --runtime-arg --log --runtime-arg=/var/lib/containers/ssd0/runroot/overlay-containers/c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8/userdata/oci-log --conmon-pidfile /var/lib/containers/ssd0/runroot/overlay-containers/c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8/userdata/conmon.pid --exit-command /usr/bin/podman --exit-command-arg --root --exit-command-arg /var/lib/containers/ssd0/storage --exit-command-arg --runroot --exit-command-arg /var/lib/containers/ssd0/runroot --exit-command-arg --log-level --exit-command-arg warning --exit-command-arg --cgroup-manager --exit-command-arg systemd --exit-command-arg --tmpdir --exit-command-arg /run/libpod --exit-command-arg --network-config-dir --exit-command-arg "" --exit-command-arg --network-backend --exit-command-arg netavark --exit-command-arg --volumepath --exit-command-arg /var/lib/containers/ssd0/storage/volumes --exit-command-arg --db-backend --exit-command-arg sqlite --exit-command-arg --transient-store=false --exit-command-arg --runtime --exit-command-arg crun --exit-command-arg --storage-driver --exit-command-arg overlay --exit-command-arg --events-backend --exit-command-arg journald --exit-command-arg container --exit-command-arg cleanup --exit-command-arg c0eb6e2299307c243d44aae6b41ac1a0a99eddb516e0b1651cd8c3935fe5b5c8 COREDUMP_CWD /run/s6/legacy-services/tdarr_server COREDUMP_ENVIRON,

Please provide the following information:

-Worker error [can be found on the 'Tdarr' tab by pressing the 'i' button on a failed item in the staged file section or in the transcode error section at the bottom]

Additional context Add any other context about the problem here.

HaveAGitGat commented 2 months ago

Do you have a screenshot of the memory graphs on the stats tab? Does that correlate with what you see externally?

Dekumori commented 1 month ago

I'm running Tdarr in Ubuntu Server 23.10 and I keep getting kernel panics that leave my system unresponsive until I reboot. This only happens when I run Tdarr. If I kill the container, then my system runs without issue.

I still need to do more testing to isolate the issue as it could be a driver issue and not specifically a Tdarr issue. I just figured I'd share my experience so far. I can share logs once I am back at home.

EDIT My issue was due to running out of swap memory. 128 GB of RAM and 8 GB of swap with swappiness set to 60 was a blunder on my part. I forgot to change the default swappiness. I adjusted the swappiness down to 10 and haven't had an issue in the past 8 hours. It's been crashing after ~12 hours.

HaveAGitGat commented 3 weeks ago

@Dekumori ty for the update