TandoorRecipes / recipes

Application for managing recipes, planning meals, building shopping lists and much much more!
https://docs.tandoor.dev
Other
5.55k stars 589 forks source link

Database not reachable on Synology NAS #1090

Closed Battlestar963 closed 2 years ago

Battlestar963 commented 2 years ago

Issue

Hello, looking at the logs it seems like the database Container is not reachable by the actual application container. I installed the containers by following the Synology Setup guide with the plain docker compose file. Following the PDF guide leads to the same error. So something must be misconfigured on my side.

Help would be greatly appreciated Thanks in advance

Setup Info

Version: latest stable one OS: Synology on latest stable Release 7.xx

.env

Please include your .env config file (make sure to remove/replace all secrets)

# only set this to true when testing/debugging
# when unset: 1 (true) - dont unset this, just for development
DEBUG=0
SQL_DEBUG=0

# hosts the application can run under e.g. recipes.mydomain.com,cooking.mydomain.com,...
ALLOWED_HOSTS=*

# random secret key, use for example `base64 /dev/urandom | head -c50` to generate one
SECRET_KEY=geifj48sj2odncjcowb2

# your default timezone See https://timezonedb.com/time-zones for a list of timezones
TIMEZONE=Europe/Berlin

# add only a database password if you want to run with the default postgres, otherwise change settings accordingly
DB_ENGINE=django.db.backends.postgresql
# DB_OPTIONS= {} # e.g. {"sslmode":"require"} to enable ssl
POSTGRES_HOST=db_recipes
POSTGRES_PORT=5432
POSTGRES_USER=djangouser
POSTGRES_PASSWORD=postgres
POSTGRES_DB=djangodb

# database connection string, when used overrides other database settings.
# format might vary depending on backend
# DATABASE_URL = engine://username:password@host:port/dbname

# the default value for the user preference 'fractions' (enable/disable fraction support)
# default: disabled=0
FRACTION_PREF_DEFAULT=0

# the default value for the user preference 'comments' (enable/disable commenting system)
# default comments enabled=1
COMMENT_PREF_DEFAULT=1

# Users can set a amount of time after which the shopping list is refreshed when they are in viewing mode
# This is the minimum interval users can set. Setting this to low will allow users to refresh very frequently which
# might cause high load on the server. (Technically they can obviously refresh as often as they want with their own scripts)
SHOPPING_MIN_AUTOSYNC_INTERVAL=5

# Default for user setting sticky navbar
# STICKY_NAV_PREF_DEFAULT=1

# If base URL is something other than just /  (you are serving a subfolder in your proxy for instance http://recipe_app/recipes/)
# SCRIPT_NAME=/recipes

# If staticfiles are stored at a different location uncomment and change accordingly, MUST END IN /
# this is not required if you are just using a subfolder
# This can either be a relative path from the applications base path or the url of an external host
# STATIC_URL=/static/

# If mediafiles are stored at a different location uncomment and change accordingly, MUST END IN /
# this is not required if you are just using a subfolder
# This can either be a relative path from the applications base path or the url of an external host
# MEDIA_URL=/media/

# Serve mediafiles directly using gunicorn. Basically everyone recommends not doing this. Please use any of the examples
# provided that include an additional nxginx container to handle media file serving.
# If you know what you are doing turn this back on (1) to serve media files using djangos serve() method.
# when unset: 1 (true) - this is temporary until an appropriate amount of time has passed for everyone to migrate
GUNICORN_MEDIA=0

# S3 Media settings: store mediafiles in s3 or any compatible storage backend (e.g. minio)
# as long as S3_ACCESS_KEY is not set S3 features are disabled
# S3_ACCESS_KEY=
# S3_SECRET_ACCESS_KEY=
# S3_BUCKET_NAME=
# S3_REGION_NAME= # default none, set your region might be required
# S3_QUERYSTRING_AUTH=1 # default true, set to 0 to serve media from a public bucket without signed urls
# S3_QUERYSTRING_EXPIRE=3600 # number of seconds querystring are valid for
# S3_ENDPOINT_URL= # when using a custom endpoint like minio

# Email Settings, see https://docs.djangoproject.com/en/3.2/ref/settings/#email-host
# Required for email confirmation and password reset (automatically activates if host is set)
# EMAIL_HOST=
# EMAIL_PORT=
# EMAIL_HOST_USER=
# EMAIL_HOST_PASSWORD=
# EMAIL_USE_TLS=0
# EMAIL_USE_SSL=0
# DEFAULT_FROM_EMAIL= # email sender address (default 'webmaster@localhost')
# ACCOUNT_EMAIL_SUBJECT_PREFIX= # prefix used for account related emails (default "[Tandoor Recipes] ")

# allow authentication via reverse proxy (e.g. authelia), leave off if you dont know what you are doing
# see docs for more information https://vabene1111.github.io/recipes/features/authentication/
# when unset: 0 (false)
REVERSE_PROXY_AUTH=0

# Default settings for spaces, apply per space and can be changed in the admin view
# SPACE_DEFAULT_MAX_RECIPES=0 # 0=unlimited recipes
# SPACE_DEFAULT_MAX_USERS=0 # 0=unlimited users per space
# SPACE_DEFAULT_MAX_FILES=0 # Maximum file storage for space in MB. 0 for unlimited, -1 to disable file upload.
# SPACE_DEFAULT_ALLOW_SHARING=1 # Allow users to share recipes with public links

# allow people to create accounts on your application instance (without an invite link)
# when unset: 0 (false)
# ENABLE_SIGNUP=0

# If signup is enabled you might want to add a captcha to it to prevent spam
# HCAPTCHA_SITEKEY=
# HCAPTCHA_SECRET=

# if signup is enabled you might want to provide urls to data protection policies or terms and conditions
# TERMS_URL=
# PRIVACY_URL=
# IMPRINT_URL=

# enable serving of prometheus metrics under the /metrics path
# ATTENTION: view is not secured (as per the prometheus default way) so make sure to secure it
# trough your web server (or leave it open of you dont care if the stats are exposed)
# ENABLE_METRICS=0

# allows you to setup OAuth providers
# see docs for more information https://vabene1111.github.io/recipes/features/authentication/
# SOCIAL_PROVIDERS = allauth.socialaccount.providers.github, allauth.socialaccount.providers.nextcloud,

# Should a newly created user from a social provider get assigned to the default space and given permission by default ?
# ATTENTION: This feature might be deprecated in favor of a space join and public viewing system in the future
# default 0 (false), when 1 (true) users will be assigned space and group
# SOCIAL_DEFAULT_ACCESS = 1

# if SOCIAL_DEFAULT_ACCESS is used, which group should be added
# SOCIAL_DEFAULT_GROUP=guest

# Django session cookie settings. Can be changed to allow a single django application to authenticate several applications
# when running under the same database
# SESSION_COOKIE_DOMAIN=.example.com
# SESSION_COOKIE_NAME=sessionid # use this only to not interfere with non unified django applications under the same top level domain

# by default SORT_TREE_BY_NAME is disabled this will store all Keywords and Food in the order they are created
# enabling this setting makes saving new keywords and foods very slow, which doesn't matter in most usecases.
# however, when doing large imports of recipes that will create new objects, can increase total run time by 10-15x
# Keywords and Food can be manually sorted by name in Admin
# This value can also be temporarily changed in Admin, it will revert the next time the application is started
# This will be fixed/changed in the future by changing the implementation or finding a better workaround for sorting
# SORT_TREE_BY_NAME=0
# LDAP authentication
# default 0 (false), when 1 (true) list of allowed users will be fetched from LDAP server
#LDAP_AUTH=
#AUTH_LDAP_SERVER_URI=
#AUTH_LDAP_BIND_DN=
#AUTH_LDAP_BIND_PASSWORD=
#AUTH_LDAP_USER_SEARCH_BASE_DN=

docker-compose.yml

When running with docker compose please provide your docker-compose.yml

version: "3"
services:
  db_recipes:
    restart: always
    image: postgres:11-alpine
    volumes:
      - ./postgresql:/var/lib/postgresql/data
    env_file:
      - ./.env

  web_recipes:
    image: vabene1111/recipes
    restart: always
    env_file:
      - ./.env
    volumes:
      - staticfiles:/opt/recipes/staticfiles
      - nginx_config:/opt/recipes/nginx/conf.d
      - ./mediafiles:/opt/recipes/mediafiles
    depends_on:
      - db_recipes

  nginx_recipes:
    image: nginx:mainline-alpine
    restart: always
    ports:
      - 2000:80
    env_file:
      - ./.env
    depends_on:
      - web_recipes
    volumes:
      - nginx_config:/etc/nginx/conf.d:ro
      - staticfiles:/static
      - ./mediafiles:/media

volumes:
  nginx_config:
  staticfiles:

Logs

If you feel like there is anything interesting please post the output of docker-compose logs at container startup and when the issue happens. Tandoor db log (paste in) Tandoor web app log Tandoor nginx log

smilerz commented 2 years ago

It looks like the DB server was restarting several times during the Tandoor startup. Can you try restarting the Tandoor container?

Battlestar963 commented 2 years ago

Unfortunately it's the same after a restart: https://pastebin.com/ELyv3w11 I only included the last 10 minutes. It was running all the time and started and stopped several workers.

koch17 commented 2 years ago

Can confirm the problem. Had to reinstall tandoor and decided to use that method and had the same problem. After it failed, stopped being lazy and ssh'ed into synology and installed it using the command line.

vabene1111 commented 2 years ago

interesting, are you seeing high CPU loads during the container start attempts. We just had a similar issue #1070 can you see if that soultion works for you as well. If yes we need to investigate why this appears to be happening on synology systems.

Battlestar963 commented 2 years ago

@koch17 Are you speaking of using docker run command for the three containers?

@vabene1111 Starting the containers one by one from the Synology UI results in no significant rise (CPU load peaks to 25% for a short time when starting web application). Starting the containers with docker compose leads to a short peak of 45%. And then it goes down to under 10%. Running docker compose with DISABLE_TREE_FIX_STARTUP=1 doesn't change the outcome. I'm still getting "is the server running on host and accepting connections". The CPU load seems unchanged.

vabene1111 commented 2 years ago

can you post logs (if you want all containers combined) when DISABLE_TREE_FIX_STARTUP is set ? Its very hard for me to understand whats going on as it seems to only affect synology devices and i dont have one but we will see what we can do to help you :)

Battlestar963 commented 2 years ago

Sure. I appreciate your time and effort:) Db log: https://pastebin.com/0eugPaWg Web app: https://pastebin.com/Nw2psnsw

I'm not bound to run it via docker compose. Its just less work than building docker run commands with all the environment variables.

Nailik commented 2 years ago

I'll update the Documentation soon, i had equal problems and managed to fix my Tandoor installation on my Synology Nas

Nailik commented 2 years ago

I had to look up my settings and the problem is, that the firewall blocks communication between the docker container. You have to enabled it in the firewall, see my pull request https://github.com/TandoorRecipes/recipes/pull/1104/commits/571a618818dbb3ebc234a58223a26f23f0618fa5

Battlestar963 commented 2 years ago

@Nailik Thank you very much. Following your updated guide the web app seems to connect to the db container.

Looks like it is sorted out. Now I need to find out why I'm not able to connect to the web interface. Neither by nginx nor by mapping a port directly on the web app. Log of web app: https://pastebin.com/1xx8qPyX Nginx log: https://pastebin.com/ZrvHMG2W

To rule out the firewall, I disabled it completely.

The website loads very long and gives an Address unreachable Error.

Nailik commented 2 years ago

I'm very sure it's a port forwarding problem also i recommend you to not turn off the firewall. For me the logs don't show any sign that you tried to reach the website.

Could you show screenshots of all the settings i mentioned in the documentation?

Otherwise you can contact me and we can figure out your problem together.

Battlestar963 commented 2 years ago

@Nailik Thanks fo your offer. It's very appreciated.

Somehow it started working after what felt like the fifth restart of the Synology. I can now connect to the web server by opening the nginx port.

Thanks everyone for your help. I'll close this issue now.