search

open-eats / OpenEats

:pizza: Self Hosted Recipe Management App :hamburger:
https://open-eats.github.io/
MIT License
669 stars 102 forks source link

Setup with external db not working (Composer errors) #155

Closed Jan-NiklasB closed 3 years ago

Jan-NiklasB commented 3 years ago

Hey, I just tried to setup openeats on my server running an MariaDB. After reading the setup guide and trying around I always get errors on running quick-setup.py

My files:

docker-prod.yml ~~~ version: '2.3' services: api: image: openeats/openeats-api command: /startup/prod-entrypoint.sh restart: on-failure volumes: - static-files:/code/static-files - site-media:/code/site-media depends_on: db: condition: service_healthy env_file: env_prod.list web: image: openeats/openeats-web command: yarn start depends_on: - api volumes: - public-ui:/code/build env_file: env_prod.list #db: # image: mariadb:5.5.64 # volumes: # - database:/var/lib/mysql # healthcheck: # test: ["CMD", "mysqladmin" ,"ping", "-h", "localhost"] # timeout: 20s # retries: 20 # env_file: # env_prod.list nginx: image: openeats/openeats-nginx command: ./start.sh volumes: - static-files:/var/www/html/openeats-static/static-files - site-media:/var/www/html/openeats-static/site-media - public-ui:/var/www/html/openeats-static/public-ui depends_on: - api - web env_file: env_prod.list volumes: #database: public-ui: static-files: site-media: ~~~
docker-prod.override.yml ~~~ version: '2.3' services: nginx: ports: - "357:80" restart: always api: restart: always web: restart: on-failure # db: # restart: always ~~~
env_prod.list ~~~ # Database config MYSQL_DATABASE=openeats MYSQL_USER=openeats MYSQL_HOST=192.168.13.123 MYSQL_PORT=3306 MQSQL_ROOT_PASSWORD=Password # Django config API_URL=0.0.0.0:8000 API_PORT=8000 DJANGO_SECRET_KEY=*** DJANGO_SETTINGS_MODULE=base.settings DJANGO_DEBUG=False ALLOWED_HOST=eat.my-domain.de HTTP_X_FORWARDED_PROTO=true # Node config NODE_ENV=production NODE_URL=localhost:8080 NODE_API_URL=https://eat.my-domain.de NODE_LOCALE=de ~~~

Error reported by quick-setup.py:

root@s02:/opt/OpenEats# python3 ./quick-start.py
==================
Downloading Images
==================
latest: Pulling from openeats/openeats-api
Digest: sha256:7d73944dc653a2781c566055c3ecd5c4d126766e256dc27dbcc36c37fae62b9a
Status: Image is up to date for openeats/openeats-api:latest
docker.io/openeats/openeats-api:latest
latest: Pulling from openeats/openeats-web
Digest: sha256:f49139cd0daa83a3420c7993c080e398c58de327032dbfaf26323ea1b4c17526
Status: Image is up to date for openeats/openeats-web:latest
docker.io/openeats/openeats-web:latest
latest: Pulling from openeats/openeats-nginx
Digest: sha256:d30864b7f37f6ad399d97f00e8f07fcef26256cf4c75facfab6f1c7517a1b117
Status: Image is up to date for openeats/openeats-nginx:latest
docker.io/openeats/openeats-nginx:latest
==================
Starting OpenEats
==================
Using remote DB...
ERROR: Service 'api' depends on service 'db' which is undefined.
ERROR: Service 'api' depends on service 'db' which is undefined.
ERROR: Service 'api' depends on service 'db' which is undefined.
ERROR: Service 'api' depends on service 'db' which is undefined.
App started. Please wait ~30 seconds for the containers to come online.

Can someone tell me what else I have to change?

Thanks in advance!

rustymyers commented 3 years ago

In the docker-prod.yml you should remove the 

    depends_on:
      db:
        condition: service_healthy

See the example here: https://github.com/open-eats/OpenEats/blob/master/docs/samples/sample_docker_prod_remote_db.yml

rustymyers commented 3 years ago

Also, check this comment for recent suggestion on getting remote DB password to work: https://github.com/open-eats/OpenEats/issues/145#issuecomment-770376367

Jan-NiklasB commented 3 years ago

@rustymyers Thanks for the fast response. I already found out thet the "depends on" has to be commented out myself.

For the password I'm still fiddeling around, as the docker compose command doesn't seeams to work for me even if I set both password variables....

my current env_prod.list

# Database config
MYSQL_DATABASE=openeats
MYSQL_USER=openeats
MYSQL_HOST=192.168.13.123
MYSQL_PORT=3306
MQSQL_ROOT_PASSWORD=PassWORD!
MYSQL_PASSWORD=PassWORD!

# Django config
API_URL=0.0.0.0:8000
API_PORT=8000
DJANGO_SECRET_KEY=****
DJANGO_SETTINGS_MODULE=base.settings
DJANGO_DEBUG=False
ALLOWED_HOST=eat.my-domain.de
HTTP_X_FORWARDED_PROTO=true

# Node config
NODE_ENV=production
NODE_URL=localhost:8080
NODE_API_URL=https://eat.my-domain.de
NODE_LOCALE=de

Console output:


root@s02:/opt/OpenEats# python3 ./quick-start.py
==================
Downloading Images
==================
latest: Pulling from openeats/openeats-api
Digest: sha256:7d73944dc653a2781c566055c3ecd5c4d126766e256dc27dbcc36c37fae62b9a
Status: Image is up to date for openeats/openeats-api:latest
docker.io/openeats/openeats-api:latest
latest: Pulling from openeats/openeats-web
Digest: sha256:f49139cd0daa83a3420c7993c080e398c58de327032dbfaf26323ea1b4c17526
Status: Image is up to date for openeats/openeats-web:latest
docker.io/openeats/openeats-web:latest
latest: Pulling from openeats/openeats-nginx
Digest: sha256:d30864b7f37f6ad399d97f00e8f07fcef26256cf4c75facfab6f1c7517a1b117
Status: Image is up to date for openeats/openeats-nginx:latest
docker.io/openeats/openeats-nginx:latest
==================
Starting OpenEats
==================
Using remote DB...
Taking a image backup save to 'site-media'...
Stopping openeats_nginx_1 ... done
Stopping openeats_api_1 ... done
Recreating openeats_api_1 ... done
Recreating openeats_web_1 ... done
Recreating openeats_nginx_1 ... done
App started. Please wait ~30 seconds for the containers to come online.
root@s02:/opt/OpenEats# mc

root@s02:/opt/OpenEats# docker-compose -f docker-prod.yml run --rm --entrypoint 'python manage.py createsuperuser' api
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 227, in get_new_connection
    return Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/__init__.py", line 86, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 204, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'openeats'@'172.18.0.3' (using password: NO)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/management/commands/createsuperuser.py", line 59, in execute
    return super().execute(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 352, in execute
    self.check_migrations()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 442, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 212, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 61, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 44, in has_table
    return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 255, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 232, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 227, in get_new_connection
    return Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/__init__.py", line 86, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 204, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (1045, "Access denied for user 'openeats'@'172.18.0.3' (using password: NO)")

The DB is accessable, as I can connect via mysql on cmd without problems....

rustymyers commented 3 years ago

Try cleaning everything out and starting fresh. https://github.com/open-eats/OpenEats/issues/147#issuecomment-866057003 (I have not tried it and I am no expert)

Jan-NiklasB commented 3 years ago

@rustymyers

Sadly no luck: (don't wonder, I already was deleting als containers and volumes)

root@s02:/opt/OpenEats# docker stop openeats_web_1 openeats_nginx_1 openeats_api_1 openeats_db_1 ; docker rm openeats_web_1 openeats_nginx_1 openeats_api_1 openeats_db_1 ; docker volume rm openeats_database
openeats_web_1
Error response from daemon: No such container: openeats_nginx_1
Error response from daemon: No such container: openeats_api_1
Error response from daemon: No such container: openeats_db_1
openeats_web_1
Error: No such container: openeats_nginx_1
Error: No such container: openeats_api_1
Error: No such container: openeats_db_1
Error: No such volume: openeats_database
root@s02:/opt/OpenEats# python3 ./quick-start.py
==================
Downloading Images
==================
latest: Pulling from openeats/openeats-api
Digest: sha256:7d73944dc653a2781c566055c3ecd5c4d126766e256dc27dbcc36c37fae62b9a
Status: Image is up to date for openeats/openeats-api:latest
docker.io/openeats/openeats-api:latest
latest: Pulling from openeats/openeats-web
Digest: sha256:f49139cd0daa83a3420c7993c080e398c58de327032dbfaf26323ea1b4c17526
Status: Image is up to date for openeats/openeats-web:latest
docker.io/openeats/openeats-web:latest
latest: Pulling from openeats/openeats-nginx
Digest: sha256:d30864b7f37f6ad399d97f00e8f07fcef26256cf4c75facfab6f1c7517a1b117
Status: Image is up to date for openeats/openeats-nginx:latest
docker.io/openeats/openeats-nginx:latest
==================
Starting OpenEats
==================
Using remote DB...
Creating volume "openeats_static-files" with default driver
Creating volume "openeats_site-media" with default driver
Creating openeats_api_1 ... done
Creating openeats_web_1 ... done
Creating openeats_nginx_1 ... done
App started. Please wait ~30 seconds for the containers to come online.
root@s02:/opt/OpenEats# docker-compose -f docker-prod.yml run --rm --entrypoint 'python manage.py createsuperuser' api
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 227, in get_new_connection
    return Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/__init__.py", line 86, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 204, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1045, "Access denied for user 'openeats'@'172.18.0.3' (using password: NO)")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 316, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.6/site-packages/django/contrib/auth/management/commands/createsuperuser.py", line 59, in execute
    return super().execute(*args, **options)
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 352, in execute
    self.check_migrations()
  File "/usr/local/lib/python3.6/site-packages/django/core/management/base.py", line 442, in check_migrations
    executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
    self.loader = MigrationLoader(self.connection)
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
    self.build_graph()
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/loader.py", line 212, in build_graph
    self.applied_migrations = recorder.applied_migrations()
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 61, in applied_migrations
    if self.has_table():
  File "/usr/local/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 44, in has_table
    return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 255, in cursor
    return self._cursor()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 232, in _cursor
    self.ensure_connection()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 216, in ensure_connection
    self.connect()
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/base/base.py", line 194, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/usr/local/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 227, in get_new_connection
    return Database.connect(**conn_params)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/__init__.py", line 86, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 204, in __init__
    super(Connection, self).__init__(*args, **kwargs2)
django.db.utils.OperationalError: (1045, "Access denied for user 'openeats'@'172.18.0.3' (using password: NO)")
Jan-NiklasB commented 3 years ago

For completenes I just changed my db user to no password, and everything works fine, except for entering the Django-backend (CSRF check failed)....

So seems like there's still a bit work to do.....

rustymyers commented 3 years ago

Try MARIADB_ROOT_PASSWORD?

Source: https://hub.docker.com/_/mariadb/

Jan-NiklasB commented 3 years ago

Try MARIADB_ROOT_PASSWORD?

Source: https://hub.docker.com/_/mariadb/

Tried this, but doesn't work either:

# Database config
MYSQL_DATABASE=openeats
MYSQL_USER=openeats
MYSQL_HOST=192.168.13.123
MYSQL_PORT=3306
MQSQL_ROOT_PASSWORD=eatME
MYSQL_PASSWORD=eatME
MARIADB_ROOT_PASSWORD=eatME
MARIADB_PASSWORD=eatME

For some reason openeats seems to don't use the password supplied....

Regarding the CSRF error, was my fault, if anyone is running into this in the future, try the option "ProxyPreserveHost On" in your revese proxy.

rustymyers commented 3 years ago

It seems the code is looking for os.environ.get('MYSQL_ROOT_PASSWORD', '') to get the password for mysql. I wonder if it can't find it, does it default to No password or empty password, or are those the same thing to mysql?

Is your password being changed by the shell? Do you have an ! or * (other special characters)? Would that effect the mysql_root_password?

Jan-NiklasB commented 3 years ago

It seems the code is looking for os.environ.get('MYSQL_ROOT_PASSWORD', '') to get the password for mysql. I wonder if it can't find it, does it default to No password or empty password, or are those the same thing to mysql?

Is your password being changed by the shell? Do you have an ! or * (other special characters)? Would that effect the mysql_root_password?

If "No Password" is equal to "empty password" depends on the constraints set in the table (as far as my smal knowledg reaches).

I already tried passwords with and without special chars (in my case "!"), so it doesn't seeams to affect the behavior... maybe the order of the variables is important to the parser?

rustymyers commented 3 years ago

I created a new instance with a custom MYSQL_ROOT_PASSWORD:

# Database config
MYSQL_DATABASE=openeats
MYSQL_ROOT_PASSWORD=rootPoop

This seems to work as expected and the database is created, as is the superuser. I don't know that this means much, but the error for connecting is still showing in my docker db container:

2021-06-22 16:42:03 57 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
2021-06-22 16:42:33 58 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
2021-06-22 16:43:03 59 [Warning] Access denied for user 'root'@'localhost' (using password: NO)

The web services are working, I can create a new recipe and log into the admin interface. I wonder if the issue isn't the database? https://github.com/open-eats/OpenEats/issues/61#issuecomment-480204592

Jan-NiklasB commented 3 years ago

So for better traceback my full setup:

OS: Debain 11 Bullseye (testing) [for hardware compatibility] Kernel 5.10.0-2amd64

Software: Webmin 1.979 Apache 2.4.46 MariaDB 10.5.8 Docker 20.10.2

Setup:

Router
    |
Server Firewall
    |---------------------------------------|
Apache (name based)                      MariaDB
    |                                       |
Docker ------------------------------------ | (direct connection via local IP)
    |
Nginx (as part of openeats)
    |
Openeats services

I hope that maybe helps to localize the issue.

Jan-NiklasB commented 3 years ago

I created a new instance with a custom MYSQL_ROOT_PASSWORD:

# Database config
MYSQL_DATABASE=openeats
MYSQL_ROOT_PASSWORD=rootPoop

This seems to work as expected and the database is created, as is the superuser. I don't know that this means much, but the error for connecting is still showing in my docker db container:

2021-06-22 16:42:03 57 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
2021-06-22 16:42:33 58 [Warning] Access denied for user 'root'@'localhost' (using password: NO)
2021-06-22 16:43:03 59 [Warning] Access denied for user 'root'@'localhost' (using password: NO)

The web services are working, I can create a new recipe and log into the admin interface. I wonder if the issue isn't the database? #61 (comment)

Well, the problem shows up when the external db hast to be populated with the initial data. The internal DB is somehow populated ignoring the password problem....

Edit: Regarding the comment, I don't think so, as at the moment I'm running openeats with a db user without PW and everything works fine, no errors at all. So I would guess it's a bug in the function to connect to the db.

rustymyers commented 3 years ago

Have you updated the docker-compose.yml file to remove the references to db? I was able to set up a phpmyadmin/mysql instance on the same host and point it to the database.

At first I got _mysql_exceptions.OperationalError: (1049, "Unknown database 'oedatabase'") from the command docker logs openeats_api_1. Then I got _mysql_exceptions.ProgrammingError: (1146, "Table 'oedatabase.django_session' doesn't exist") and did the migration with the following commands:

docker exec -it openeats_api_1 /bin/sh
$ python manage.py makemigrations
$ python manage.py migrate
$ python manage.py createsuperuser
Jan-NiklasB commented 3 years ago

Have you updated the docker-compose.yml file to remove the references to db? I was able to set up a phpmyadmin/mysql instance on the same host and point it to the database.

At first I got _mysql_exceptions.OperationalError: (1049, "Unknown database 'oedatabase'") from the command docker logs openeats_api_1. Then I got _mysql_exceptions.ProgrammingError: (1146, "Table 'oedatabase.django_session' doesn't exist") and did the migration with the following commands:

docker exec -it openeats_api_1 /bin/sh
$ python manage.py makemigrations
$ python manage.py migrate
$ python manage.py createsuperuser

Yes, i did.

In my log, the error is the same as posted above.

Any command including a call to the DB ended in "Access denied" until I changed the user to not require a authentication.

After this, I did the steps you mentioned with the migration, which worked as they should.

rustymyers commented 3 years ago

Try updating your env_prod.list to use MYSQL_ROOT_PASSWORD instead of MQSQL_ROOT_PASSWORD

Jan-NiklasB commented 3 years ago

Try updating your env_prod.list to use MYSQL_ROOT_PASSWORD instead of MQSQL_ROOT_PASSWORD

Wow......

Sorry for wasting your time by this stupid mistake....

Everthing works now as exspected after applying the migrations after the restart again -.-

Thank you very much for your help!

rustymyers commented 3 years ago

Sorry it took me so long to see that :)