search

meeb / tubesync

Syncs YouTube channels and playlists to a locally hosted media server
GNU Affero General Public License v3.0
1.94k stars 123 forks source link

Tasks are scheduled and never run on Docker #231

Open DougEubanks opened 2 years ago

DougEubanks commented 2 years ago

I've been trying to archive dozens of channels, and I've run into recurring issues with tasks that are scheduled to run immediately and just stop. I've tried multiple versions of the official docker image, I've built my own image using the latest main code and the offical Dockerfile. I've tried resetting the tasks multiple times, wiping the DB and output files, etc and I still end up in this state sooner or later.

I migrated started over with a MySQL based DB, and I'm at the same problem. I don't see any errors in the logs.

[services.d] starting services 243:C 07 Apr 2022 09:52:42.548 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 243:C 07 Apr 2022 09:52:42.548 # Redis version=6.0.16, bits=64, commit=00000000, modified=0, pid=243, just started 243:C 07 Apr 2022 09:52:42.548 # Configuration loaded 243:M 07 Apr 2022 09:52:42.549 * Running mode=standalone, port=6379. 243:M 07 Apr 2022 09:52:42.550 # Server initialized 243:M 07 Apr 2022 09:52:42.550 * Ready to accept connections [services.d] done. [2022-04-07 09:52:42 -0400] [242] [INFO] Starting gunicorn 20.1.0 [2022-04-07 09:52:42 -0400] [242] [INFO] Listening at: http://127.0.0.1:8080 (242) [2022-04-07 09:52:42 -0400] [242] [INFO] Using worker: sync [2022-04-07 09:52:42 -0400] [275] [INFO] Booting worker with pid: 275 2022-04-07 09:52:42,767 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync [2022-04-07 09:52:42 -0400] [276] [INFO] Booting worker with pid: 276 [2022-04-07 09:52:42 -0400] [277] [INFO] Booting worker with pid: 277 [2022-04-07 09:52:42 -0400] [279] [INFO] Booting worker with pid: 279 [2022-04-07 09:52:43 -0400] [280] [INFO] Booting worker with pid: 280 [2022-04-07 09:52:43 -0400] [283] [INFO] Booting worker with pid: 283 [2022-04-07 09:52:43 -0400] [282] [INFO] Booting worker with pid: 282 [2022-04-07 09:52:43 -0400] [285] [INFO] Booting worker with pid: 285 [2022-04-07 09:52:43 -0400] [284] [INFO] Booting worker with pid: 284 2022-04-07 09:52:43,461 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:43,862 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:44,173 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:44,484 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:44,968 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:45,075 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:45,466 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:45,571 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 2022-04-07 09:52:45,579 [tubesync/INFO] Using database connection: django.db.backends.mysql://tubesync:[hidden]@192.168.250.253:3306/tubesync 192.168.10.22 - - [07/Apr/2022:09:59:54 -0400] "GET / HTTP/1.1" 200 3618 "http://192.168.250.25/tasks" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:55 -0400] "GET /static/styles/tubesync.css HTTP/1.1" 200 34505 "http://192.168.250.25/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:55 -0400] "GET /static/fonts/roboto/roboto-regular.woff HTTP/1.1" 304 0 "http://192.168.250.25/static/styles/tubesync.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:55 -0400] "GET /static/fonts/fontawesome/fa-brands-400.woff2 HTTP/1.1" 304 0 "http://192.168.250.25/static/styles/tubesync.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:55 -0400] "GET /static/fonts/fontawesome/fa-solid-900.woff2 HTTP/1.1" 304 0 "http://192.168.250.25/static/styles/tubesync.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:55 -0400] "GET /static/fonts/fontawesome/fa-regular-400.woff2 HTTP/1.1" 304 0 "http://192.168.250.25/static/styles/tubesync.css" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:55 -0400] "GET /static/images/favicon.ico HTTP/1.1" 304 0 "http://192.168.250.25/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36" 192.168.10.22 - - [07/Apr/2022:09:59:58 -0400] "GET /tasks HTTP/1.1" 200 3521 "http://192.168.250.25/" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.83 Safari/537.36"

Screenshots are included of my current state. download (6) download (5)

DougEubanks commented 2 years ago

I've dug through the current issues and I've done all I can to debug this on my own. It doesn't matter if it's a vanilla docker image running on unRAID running in containerd on my Kubernetes cluster. I just don't know where to go to troubleshoot this next. I've spent weeks trying to reproduce and resolve this error.

meeb commented 2 years ago

TubeSync is really just a wrapper around yt-dlp (and formerly youtube-dl). That error effectively means "a download was started for a media item, and when yt-dlp completed the download, the expected output file did not exist". It's a generic "download failed" sort of error. This is generally caused by a couple of types of issues:

  1. Permissions issues writing the output file
  2. The video not being properly available at YouTube
  3. ffmpeg is failing to combine downloaded media streams into the out combined output file, this is a known issue with ffmpeg on some videos such that there are custom builds of ffmpeg which contain a patch to resolve the issue

If your issues fall into number 3. it's an upstream issue I've been tracking but could still be impacting you, you could try changing the desired source codecs though. Does issue #181 match your experience?

I really didn't want to have to start bundling in custom builds of ffmpeg, at the moment TubeSync uses the default Debian packaged ffmpeg.

meeb commented 2 years ago

Oh as for your 14 tasks that are scheduled, are they not running? Can you drop a shell into the container and check the process_tasks worker is running?

meeb commented 2 years ago

@tamahau30494 you can always also just view processes in containers as root from the host machine as well, it doesn't have to be in the container. If you want to check processes in the container it's just Debian, so you can install ps if you like. Drop a root shell into the container and it's just:

$ apt update && apt install procps

Obviously this is not persistent if you remove and recreate the container. Things like ps and many other tools are not installed in many containers to keep their size down. TubeSync is a pretty chonky container as it is given it needs a big Python stack and ffpmeg installed.

meeb commented 2 years ago

@tamahau30494 sure, as no processes run as root in the container. Try:

$ ps aux

But, this is is a level of debugging that's probably not especially useful for you directly, probably restarting the container and running the reset-tasks CLI command is going to be more effective.

meeb commented 2 years ago

Yep, but if that button times out (it can if you have a load of sources or media added as it's an HTTP request) you can follow this guide to reset tasks with no time limit off the docker host:

https://github.com/meeb/tubesync/blob/main/docs/reset-tasks.md