TandoorRecipes / recipes

Application for managing recipes, planning meals, building shopping lists and much much more!
https://docs.tandoor.dev
Other
5.34k stars 563 forks source link

(QA Team here to help you) DB server continue restarting, authentications issues. #2608

Closed jc-ag closed 1 year ago

jc-ag commented 1 year ago

Tandoor Version

NA

Setup

Docker / Docker-Compose

Reverse Proxy

No reverse proxy

Other

Docker Compose - Plain (https://docs.tandoor.dev/install/docker/#docker-compose)

Bug description

Hi guys! Not sure if it's the correct place to put this info. I'm a QA Engineer and I'm trying to get some close people into the IT world (as QA too of course). We're looking for an Open Source project to collaborate with.

Our plan is help on a repo/project as an independent QA team with me leading them. During the process I will teach them what is to test a software under real conditions.

I thought your project is the perfect candidate to do this. I guess it would be a gain-gain effort since you guys will get dedicated QA's.

For the plan, we though we could include:

We still have to organize people, tools, process, etc, but we would like to hear your thoughts on this.


BUG:

To start the initiative mentioned above I was trying to start an environment locally and do some exploratory navigation through the procedure explained here, specifically in the Docker Compose - Plain part.

Doing docker compose up -d seems to work fine, but the web_recipes service begins restarting continuously and cannot load the application. For what I've tried, seems the DB service described in the Docker Compose file is not taking any value from the .env file.

I've tried even loading only the db service using your docker-compose.yml + .env and I'm not able to connect to DB through pgAdmin (I've exposed 5432 to do so in the .yml)


docker-compose.yml


version: "3"
services:
  db_recipes:
    restart: always
    ports:
      - 5432:5432
    image: postgres:15-alpine
    volumes:
      - ./postgresql:/var/lib/postgresql/data
    env_file:
      - ./.env

  web_recipes:
    image: vabene1111/recipes
    env_file:
      - ./.env
    volumes:
      - staticfiles:/opt/recipes/staticfiles
      # Do not make this a bind mount, see https://docs.tandoor.dev/install/docker/#volumes-vs-bind-mounts
      - nginx_config:/opt/recipes/nginx/conf.d
      - ./mediafiles:/opt/recipes/mediafiles
    depends_on:
      - db_recipes

  nginx_recipes:
    image: nginx:mainline-alpine
    restart: always
    ports:
      - 8081:80
    env_file:
      - ./.env
    depends_on:
      - web_recipes
    volumes:
      # Do not make this a bind mount, see https://docs.tandoor.dev/install/docker/#volumes-vs-bind-mounts
      - nginx_config:/etc/nginx/conf.d:ro
      - staticfiles:/static:ro
      - ./mediafiles:/media:ro

volumes:
  nginx_config:
  staticfiles:

.env

# only set this to true when testing/debugging
# when unset: 1 (true) - dont unset this, just for development
DEBUG=0
SQL_DEBUG=0
DEBUG_TOOLBAR=0
# Gunicorn log level for debugging (default value is "info" when unset)
# (see https://docs.gunicorn.org/en/stable/settings.html#loglevel for available settings)
# GUNICORN_LOG_LEVEL="debug"

# HTTP port to bind to
# TANDOOR_PORT=8080

# hosts the application can run under e.g. recipes.mydomain.com,cooking.mydomain.com,...
ALLOWED_HOSTS=*

# random secret key, use for example `base64 /dev/urandom | head -c50` to generate one
# ---------------------------- REQUIRED -------------------------
SECRET_KEY=djangodb
# ---------------------------------------------------------------

# your default timezone See https://timezonedb.com/time-zones for a list of timezones
TIMEZONE=Europe/Berlin

# add only a database password if you want to run with the default postgres, otherwise change settings accordingly
DB_ENGINE=django.db.backends.postgresql
# DB_OPTIONS= {} # e.g. {"sslmode":"require"} to enable ssl
POSTGRES_HOST=db_recipes
POSTGRES_PORT=5432
POSTGRES_USER=djangouser
# ---------------------------- REQUIRED -------------------------
POSTGRES_PASSWORD=djangopassword
# ---------------------------------------------------------------
POSTGRES_DB=djangodb

# database connection string, when used overrides other database settings.
# format might vary depending on backend
# DATABASE_URL = engine://username:password@host:port/dbname

# the default value for the user preference 'fractions' (enable/disable fraction support)
# default: disabled=0
FRACTION_PREF_DEFAULT=0

# the default value for the user preference 'comments' (enable/disable commenting system)
# default comments enabled=1
COMMENT_PREF_DEFAULT=1

# Users can set a amount of time after which the shopping list is refreshed when they are in viewing mode
# This is the minimum interval users can set. Setting this to low will allow users to refresh very frequently which
# might cause high load on the server. (Technically they can obviously refresh as often as they want with their own scripts)
SHOPPING_MIN_AUTOSYNC_INTERVAL=5

# Default for user setting sticky navbar
# STICKY_NAV_PREF_DEFAULT=1

# If base URL is something other than just / (you are serving a subfolder in your proxy for instance http://recipe_app/recipes/)
# Be sure to not have a trailing slash: e.g. '/recipes' instead of '/recipes/'
# SCRIPT_NAME=/recipes

# If staticfiles are stored at a different location uncomment and change accordingly, MUST END IN /
# this is not required if you are just using a subfolder
# This can either be a relative path from the applications base path or the url of an external host
# STATIC_URL=/static/

# If mediafiles are stored at a different location uncomment and change accordingly, MUST END IN /
# this is not required if you are just using a subfolder
# This can either be a relative path from the applications base path or the url of an external host
# MEDIA_URL=/media/

# Serve mediafiles directly using gunicorn. Basically everyone recommends not doing this. Please use any of the examples
# provided that include an additional nxginx container to handle media file serving.
# If you know what you are doing turn this back on (1) to serve media files using djangos serve() method.
# when unset: 1 (true) - this is temporary until an appropriate amount of time has passed for everyone to migrate
GUNICORN_MEDIA=0

# GUNICORN SERVER RELATED SETTINGS (see https://docs.gunicorn.org/en/stable/design.html#how-many-workers for recommended settings)
# GUNICORN_WORKERS=1
# GUNICORN_THREADS=1

# S3 Media settings: store mediafiles in s3 or any compatible storage backend (e.g. minio)
# as long as S3_ACCESS_KEY is not set S3 features are disabled
# S3_ACCESS_KEY=
# S3_SECRET_ACCESS_KEY=
# S3_BUCKET_NAME=
# S3_REGION_NAME= # default none, set your region might be required
# S3_QUERYSTRING_AUTH=1 # default true, set to 0 to serve media from a public bucket without signed urls
# S3_QUERYSTRING_EXPIRE=3600 # number of seconds querystring are valid for
# S3_ENDPOINT_URL= # when using a custom endpoint like minio
# S3_CUSTOM_DOMAIN= # when using a CDN/proxy to S3 (see https://github.com/TandoorRecipes/recipes/issues/1943)

# Email Settings, see https://docs.djangoproject.com/en/3.2/ref/settings/#email-host
# Required for email confirmation and password reset (automatically activates if host is set)
# EMAIL_HOST=
# EMAIL_PORT=
# EMAIL_HOST_USER=
# EMAIL_HOST_PASSWORD=
# EMAIL_USE_TLS=0
# EMAIL_USE_SSL=0
# email sender address (default 'webmaster@localhost')
# DEFAULT_FROM_EMAIL=
# prefix used for account related emails (default "[Tandoor Recipes] ")
# ACCOUNT_EMAIL_SUBJECT_PREFIX=

# allow authentication via the REMOTE-USER header (can be used for e.g. authelia).
# ATTENTION: Leave off if you don't know what you are doing! Enabling this without proper configuration will enable anybody
#   to login with any username!
# See docs for additional information: https://docs.tandoor.dev/features/authentication/#reverse-proxy-authentication
# when unset: 0 (false)
REMOTE_USER_AUTH=0

# Default settings for spaces, apply per space and can be changed in the admin view
# SPACE_DEFAULT_MAX_RECIPES=0 # 0=unlimited recipes
# SPACE_DEFAULT_MAX_USERS=0 # 0=unlimited users per space
# SPACE_DEFAULT_MAX_FILES=0 # Maximum file storage for space in MB. 0 for unlimited, -1 to disable file upload.
# SPACE_DEFAULT_ALLOW_SHARING=1 # Allow users to share recipes with public links

# allow people to create accounts on your application instance (without an invite link)
# when unset: 0 (false)
# ENABLE_SIGNUP=0

# If signup is enabled you might want to add a captcha to it to prevent spam
# HCAPTCHA_SITEKEY=
# HCAPTCHA_SECRET=

# if signup is enabled you might want to provide urls to data protection policies or terms and conditions
# TERMS_URL=
# PRIVACY_URL=
# IMPRINT_URL=

# enable serving of prometheus metrics under the /metrics path
# ATTENTION: view is not secured (as per the prometheus default way) so make sure to secure it
# trough your web server (or leave it open of you dont care if the stats are exposed)
# ENABLE_METRICS=0

# allows you to setup OAuth providers
# see docs for more information https://docs.tandoor.dev/features/authentication/
# SOCIAL_PROVIDERS = allauth.socialaccount.providers.github, allauth.socialaccount.providers.nextcloud,

# Should a newly created user from a social provider get assigned to the default space and given permission by default ?
# ATTENTION: This feature might be deprecated in favor of a space join and public viewing system in the future
# default 0 (false), when 1 (true) users will be assigned space and group
# SOCIAL_DEFAULT_ACCESS = 1

# if SOCIAL_DEFAULT_ACCESS is used, which group should be added
# SOCIAL_DEFAULT_GROUP=guest

# Django session cookie settings. Can be changed to allow a single django application to authenticate several applications
# when running under the same database
# SESSION_COOKIE_DOMAIN=.example.com
# SESSION_COOKIE_NAME=sessionid # use this only to not interfere with non unified django applications under the same top level domain

# by default SORT_TREE_BY_NAME is disabled this will store all Keywords and Food in the order they are created
# enabling this setting makes saving new keywords and foods very slow, which doesn't matter in most usecases.
# however, when doing large imports of recipes that will create new objects, can increase total run time by 10-15x
# Keywords and Food can be manually sorted by name in Admin
# This value can also be temporarily changed in Admin, it will revert the next time the application is started
# This will be fixed/changed in the future by changing the implementation or finding a better workaround for sorting
# SORT_TREE_BY_NAME=0
# LDAP authentication
# default 0 (false), when 1 (true) list of allowed users will be fetched from LDAP server
#LDAP_AUTH=
#AUTH_LDAP_SERVER_URI=
#AUTH_LDAP_BIND_DN=
#AUTH_LDAP_BIND_PASSWORD=
#AUTH_LDAP_USER_SEARCH_BASE_DN=
#AUTH_LDAP_TLS_CACERTFILE=
#AUTH_LDAP_START_TLS=

# Enables exporting PDF (see export docs)
# Disabled by default, uncomment to enable
# ENABLE_PDF_EXPORT=1

# Recipe exports are cached for a certain time by default, adjust time if needed
# EXPORT_FILE_CACHE_DURATION=600

Relevant logs


Docker container logs (DB):

recipes-web_recipes-1
vabene1111/recipes
EXITED (1)

Checking configuration...

Waiting for database to be ready...

Database is ready

Migrating database

Traceback (most recent call last):

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 282, in ensure_connection

    self.connect()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner

    return func(*args, **kwargs)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 263, in connect

    self.connection = self.get_new_connection(conn_params)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner

    return func(*args, **kwargs)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 215, in get_new_connection

    connection = Database.connect(**conn_params)

  File "/opt/recipes/venv/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect

    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)

psycopg2.OperationalError: FATAL:  password authentication failed for user "djangouser"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/opt/recipes/manage.py", line 15, in <module>

    execute_from_command_line(sys.argv)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line

    utility.execute()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/core/management/__init__.py", line 420, in execute

    django.setup()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/__init__.py", line 24, in setup

    apps.populate(settings.INSTALLED_APPS)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/apps/registry.py", line 124, in populate

    app_config.ready()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django_prometheus/apps.py", line 24, in ready

    ExportMigrations()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django_prometheus/migrations.py", line 52, in ExportMigrations

    executor = MigrationExecutor(connections[alias])

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/migrations/executor.py", line 18, in __init__

    self.loader = MigrationLoader(self.connection)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/migrations/loader.py", line 58, in __init__

    self.build_graph()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/migrations/loader.py", line 235, in build_graph

    self.applied_migrations = recorder.applied_migrations()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/migrations/recorder.py", line 81, in applied_migrations

    if self.has_table():

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/migrations/recorder.py", line 57, in has_table

    with self.connection.cursor() as cursor:

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner

    return func(*args, **kwargs)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 323, in cursor

    return self._cursor()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 299, in _cursor

    self.ensure_connection()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner

    return func(*args, **kwargs)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 281, in ensure_connection

    with self.wrap_database_errors:

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__

    raise dj_exc_value.with_traceback(traceback) from exc_value

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 282, in ensure_connection

    self.connect()

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner

    return func(*args, **kwargs)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/base/base.py", line 263, in connect

    self.connection = self.get_new_connection(conn_params)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/utils/asyncio.py", line 26, in inner

    return func(*args, **kwargs)

  File "/opt/recipes/venv/lib/python3.10/site-packages/django/db/backends/postgresql/base.py", line 215, in get_new_connection

    connection = Database.connect(**conn_params)

  File "/opt/recipes/venv/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect

    conn = _connect(dsn, connection_factory=connection_factory, **kwasync)

django.db.utils.OperationalError: FATAL:  password authentication failed for user "djangouser"
jc-ag commented 1 year ago

Managed to make it work. The DB is not being created automatically, I had to created it manually through pgAdmin. To do that I had to open the connection rules on pg_hba.conf:

# host all all all scram-sha-256
host    all       all    0.0.0.0/0   trust

Then I connected to the container and created the DB manually and all started to work.

Probably the script that takes the .env variables and do this part is ignoring the variable for some reason?

NOTE: I'm doing this in a Macbook Air M1

vabene1111 commented 1 year ago

that is rather strange, with the default configuration everything should be created automatically by the pg container, but glad you got it working