Frequent 500 internal server errors (Failed to retrieve tasks. Database Unreachable.)

[Casuallynoted]

I'm currently running tubesync via docker, accessing via the local IP and port. I've noticed that more often than not when I've refreshed a page or navigated to a different page, it throws a 500 internal server error. I can refresh and eventually it'll load, but it oftentimes takes several tries before it loads again. This issue does not happen when no sources have been added. After adding a source, it happens nonstop. I'm noticing a "Failed to retrieve tasks. Database Unreachable." in the console multiple times over.

[meeb]

Is it a large source with a load of videos? Are you using the SQLite default database? Where is the SQLite database stored? Locally, or on a network drive?

Basically write contention to the database is a common issue with large channels or if the database is stored on a network share, you can solve it by switching to a heavier backend like Postgres.

[Extarys]

I also started getting 500 errors. I updated to the latest version but it was still happening, hence my presence on GitHub :laughing:

HDD is a WD black, attached directly to the computer (SATA3) running TubeSync.

I deleted the tubesync folder, restarted the container and added a single source. Now when I try to edit that source while tubesync is indexing I get 500 errors.

Database size is 2.8 MB. Portainer config is simple:

version: "3.0"
services:
  tubesync:
    image: ghcr.io/meeb/tubesync:latest
    container_name: tubesync
    ports:
      - 4848:4848
    volumes:
      - /media/HDD/.containers/tubesync/config:/config
      - /media/NAS/Youtube/:/downloads
    environment:
      - TZ=America/Toronto
      - PUID=1000
      - PGID=1000

Not quite sure what changed but I never had this issue before :/ Strange.

[meeb]

Anything useful in the logs? docker logs tubesync

[Extarys]

Oh :man_facepalming: here you go:

2021-05-17 11:33:04,829 [tubesync/INFO] Indexed media: Channel... / vidid
2021-05-17 11:33:04,943 [tubesync/WARNING] Media: Channel... / vidid has no published date set, marking to be skipped
2021-05-17 11:33:05,046 [tubesync/INFO] Scheduling task to download metadata for: https://www.youtube.com/watch?v=vidid
Failed to retrieve tasks. Database unreachable.
2021-05-17 11:33:05,188 [tubesync/INFO] Indexed media: Channel... / vidid
2021-05-17 11:33:05,296 [tubesync/WARNING] Media: Channel... / vidid has no published date set, marking to be skipped
2021-05-17 11:33:05,501 [tubesync/INFO] Scheduling task to download metadata for: https://www.youtube.com/watch?v=vidid
2021-05-17 11:33:08,754 [tubesync/INFO] Indexed media: Channel... / vidid
2021-05-17 11:33:08,862 [tubesync/WARNING] Media: Channel... / vidid has no published date set, marking to be skipped
[2021-05-17 11:33:08 -0400] [225] [CRITICAL] WORKER TIMEOUT (pid:4874)

EDIT: Not sure what happened but while I hit F5 to have an error for the logs, I got one 502 http code :thinking:

[meeb]

Thanks, what request are you making to tubesync when the 500 occurs? Like /whaeverurl

[Extarys]

Latest url was http://127.0.0.1:4848/sources, that one I used to have the error above.

But yesterday I had much trouble saving and deleting sources as both the following gave a 500 error:

• edit a source's settings, once I pressed the submit button on http://127.0.0.1:4848/source-update/:uuid
• and when I pressed "Really delete source" on the page http://127.0.0.1:4848/source-delete/:uuid

[meeb]

Yeah deleting sources and saving source settings are both currently very slow operations and there's not much that can be done to speed them up. Before the 1.0 release the work done deleting and updating a source will be moved to background workers (tracked in #93).

[meeb]

I'm going to close this issue as it should be resolved naturally when other issues are resolved. It's a semi-duplicate issue. Once the 1.0 release is out if you still experience any HTTP level errors, 500 or otherwise, please feel free to create another issue.