bitmagnet-io / bitmagnet

A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent search engine with web UI, GraphQL API and Servarr stack integration.
https://bitmagnet.io/
MIT License
2.28k stars 89 forks source link

Container starts before database is ready #64

Open askiiart opened 9 months ago

askiiart commented 9 months ago

Have you checked the roadmap on the website, and existing issues, before opening a dupllcate issue? Yes

Describe the bug An SQL error is shown when TMDB_API_KEY is not provided.

To Reproduce Steps to reproduce the behavior:

  1. Create new bitmagnet instance without an API key
  2. Go to the web UI.

Expected behavior It should have a more descriptive error message that says that TMDB_API_KEY is missing.

General (please complete the following information):

Additional context

Screenshot:

Snapshot_2023-11-15_13-39-44

mgdigital commented 9 months ago

Hi, this error isn't related to the TMDB API key - looks like an issue with your backend DB connection, either the migrations didn't run, a table got dropped or there's a connection issue. Can you see any tables or data in your DB, especially the torrent_contents table?

Send8213 commented 9 months ago

Describe the bug

I can reproduce a similar error when starting it up with a near stock compose file with no existing data. When it happens, the bitmagnet database does not exist in postgres when checked directly (e.g. \c bitmagnet; \d), even after waiting 5+ minutes. Restarting with compose down and up a few times does seem to fix it for me.

Could the bitmagnet service be trying to access and use the database before it has been fully setup?

To Reproduce

  1. Up the compose file with no existing data
  2. Check that bitmagnet database is not created
  3. Restart using docker compose down and up
  4. Not all tables are present still
  5. Restart again
  6. Everything seems to be working and there are no database errors in the logs

Expected behavior The database should be set up and ready to use upon first start, not needing repeated restarts.

General (please complete the following information):

Additional context Tested with both the build: and image: versions.

Changes to docker-compose.yml from docker-compose.dev.yml: bind mounts for all three containers changed to named volumes.

Selection of logs from the database via (I seem to be able to recreate the pattern consistently): docker logs bitmagnet-postgres

First startup: LOG: database system is ready to accept connections ERROR: relation "torrents" does not exist at character 177

Second startup: LOG: database system is ready to accept connections ERROR: relation "goose_db_version" does not exist at character 36

Third startup: LOG: database system is ready to accept connections (no errors)

mgdigital commented 9 months ago

Hi, looks like this is due to the Postgres health check: pg_isready reports that the DB server is ready to accept connections, but doesn't guarantee that the database defined in POSTGRES_DB has been created yet.

There's a possible workaround here: https://stackoverflow.com/questions/14549270/check-if-database-exists-in-postgresql-using-shell/16783253#16783253 but it seems convoluted - there must be a better way. Perhaps it should be the Bitmagnet container's job to attempt DB creation if it doesn't exist and we'd remove the POSTGRES_DB env var, but I'm not sure I like that either. The POSTGRES_DB env var on that Docker image has surely been provided for this purpose but then they've provided no simple way to ensure a smooth startup...