search

mediacms-io / mediacms

MediaCMS is a modern, fully featured open source video and media CMS, written in Python/Django and React, featuring a REST API.
https://mediacms.io
GNU Affero General Public License v3.0
2.74k stars 507 forks source link

Docker Compose version never seems to fully launch #934

Open tokenwizard opened 10 months ago

tokenwizard commented 10 months ago

Describe the issue I followed the Docker Compose installation steps and it seems like the services are starting up, but they just seem to be indefinitely launching and never completely running.

To Reproduce Steps to reproduce the issue:

  1. Follow the guide here: https://github.com/mediacms-io/mediacms/blob/main/docs/admins_docs.md#3-docker-installation
  2. run docker-compose up
  3. wait...wait...and wait some more. The mediacms_web container never seems to run. So far it has been about 10 minutes and the output is still as you see below.

Expected behavior Expect all services to start up and be accessible at http://SERVER_IP

Screenshots

root@HomelabTV:~/mediacms# docker-compose up
WARNING: The yTR3 variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_DB variable is not set. Defaulting to a blank string.
WARNING: The POSTGRES_USER variable is not set. Defaulting to a blank string.
Creating mediacms_redis_1 ... done
Creating mediacms_db_1    ... done
Creating mediacms_celery_beat_1 ... done
Creating mediacms_migrations_1  ... done
Creating mediacms_celery_worker_1 ... done
Creating mediacms_web_1           ... done
Attaching to mediacms_db_1, mediacms_redis_1, mediacms_celery_beat_1, mediacms_migrations_1, mediacms_web_1, mediacms_celery_worker_1
db_1             | 
db_1             | PostgreSQL Database directory appears to contain a database; Skipping initialization
db_1             | 
db_1             | 2023-12-05 14:25:56.336 GMT [1] LOG:  starting PostgreSQL 15.2 on x86_64-pc-linux-musl, compiled by gcc (Alpine 12.2.1_git20220924-r4) 12.2.1 20220924, 64-bit
db_1             | 2023-12-05 14:25:56.336 GMT [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db_1             | 2023-12-05 14:25:56.336 GMT [1] LOG:  listening on IPv6 address "::", port 5432
db_1             | 2023-12-05 14:25:56.357 GMT [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
db_1             | 2023-12-05 14:25:56.390 GMT [23] LOG:  database system was shut down at 2023-12-05 14:21:34 GMT
db_1             | 2023-12-05 14:25:56.411 GMT [1] LOG:  database system is ready to accept connections
db_1             | 2023-12-05 14:26:06.264 GMT [33] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:26:16.563 GMT [40] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:26:27.099 GMT [48] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:26:37.641 GMT [55] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:26:48.223 GMT [62] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:26:58.660 GMT [72] FATAL:  role "root" does not exist
redis_1          | 1:C 05 Dec 2023 14:25:56.166 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
redis_1          | 1:C 05 Dec 2023 14:25:56.166 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis_1          | 1:C 05 Dec 2023 14:25:56.166 * Redis version=7.2.3, bits=64, commit=00000000, modified=0, pid=1, just started
redis_1          | 1:C 05 Dec 2023 14:25:56.166 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis_1          | 1:M 05 Dec 2023 14:25:56.167 * monotonic clock: POSIX clock_gettime
redis_1          | 1:M 05 Dec 2023 14:25:56.168 * Running mode=standalone, port=6379.
redis_1          | 1:M 05 Dec 2023 14:25:56.168 * Server initialized
redis_1          | 1:M 05 Dec 2023 14:25:56.168 * Ready to accept connections tcp
db_1             | 2023-12-05 14:27:09.356 GMT [80] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:27:19.934 GMT [87] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:27:30.234 GMT [95] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:27:40.684 GMT [102] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:27:51.194 GMT [109] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:28:01.604 GMT [117] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:28:12.332 GMT [124] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:28:22.814 GMT [131] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:28:33.001 GMT [139] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:28:43.220 GMT [146] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:28:53.736 GMT [152] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:29:04.305 GMT [160] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:29:14.698 GMT [167] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:29:25.235 GMT [174] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:29:35.679 GMT [181] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:29:46.149 GMT [188] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:29:56.449 GMT [194] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:30:06.979 GMT [202] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:30:17.460 GMT [209] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:30:27.979 GMT [217] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:30:38.495 GMT [224] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:30:48.974 GMT [232] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:30:56.487 GMT [21] LOG:  checkpoint starting: time
db_1             | 2023-12-05 14:30:57.430 GMT [21] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.001 s, sync=0.520 s, total=0.943 s; sync files=2, longest=0.330 s, average=0.260 s; distance=0 kB, estimate=0 kB
db_1             | 2023-12-05 14:30:59.348 GMT [240] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:31:09.962 GMT [246] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:31:20.342 GMT [253] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:31:30.660 GMT [262] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:31:41.310 GMT [269] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:31:51.874 GMT [276] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:32:02.288 GMT [285] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:32:12.649 GMT [292] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:32:22.839 GMT [298] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:32:33.288 GMT [306] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:32:43.646 GMT [313] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:32:53.962 GMT [320] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:33:04.303 GMT [328] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:33:14.712 GMT [334] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:33:25.078 GMT [341] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:33:35.417 GMT [348] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:33:45.735 GMT [356] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:33:56.226 GMT [363] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:34:06.767 GMT [371] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:34:17.239 GMT [378] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:34:27.779 GMT [385] FATAL:  role "root" does not exist
db_1             | 2023-12-05 14:34:38.182 GMT [392] FATAL:  role "root" does not exist

Environment (please complete the following information):

tokenwizard commented 10 months ago

Just as an update, I left that running for two hours and other than additional lines of output from db_1 about the root user, there is no progress. I never get any console output from the web_1 container, but also no errors.

johann-taberlet commented 10 months ago

I'm in the same boat. Here are my logs:

mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 0 mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-db-1 | 2023-12-07 20:50:18.603 GMT [64] FATAL: role "root" does not exist mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-db-1 | 2023-12-07 20:50:28.677 GMT [71] FATAL: role "root" does not exist mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-db-1 | 2023-12-07 20:50:38.746 GMT [79] FATAL: role "root" does not exist mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.pack': Permission denied mediacms-migrations-1 | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-25ede325a6de19191c8a51f31995ac6967b074af.idx': Permission denied mediacms-migrations-1 exited with code 1 mediacms-db-1 | 2023-12-07 20:50:48.815 GMT [86] FATAL: role "root" does not exist mediacms-db-1 | 2023-12-07 20:50:58.880 GMT [94] FATAL: role "root" does not exist mediacms-db-1 | 2023-12-07 20:51:08.942 GMT [101] FATAL: role "root" does not exist

Web app service never starts. I've seen your comment in #894 but in this case it is not just spam.

MacOS silicon Docker Desktop Git clone => docker compose up

vijaykumar1710 commented 9 months ago

same issue.. Did anyone fix it? @johann-taberlet @tokenwizard

lin-yifan1 commented 9 months ago

As mentioned in https://github.com/mediacms-io/mediacms/issues/894#issuecomment-1748529012, this is just a message that can be ignored. You should be able to access http://localhost while the console is still printing these logs. However, if you want to remove these logs, one solution is to add the PGUSER variable in the docker-compose.yaml file：

db:
    image: postgres:15.2-alpine
    volumes:
      - ../postgres_data:/var/lib/postgresql/data/
    restart: always
    environment:
      POSTGRES_USER: mediacms
      POSTGRES_PASSWORD: mediacms
      POSTGRES_DB: mediacms
      PGUSER: mediacms # add here
      ...

Other solutions are mentioned here.

vijaykumar1710 commented 9 months ago

@lin-yifan1 I have one more issue while running this project via docker on my macbook.

Below is the docker logs while running it. Do you know how to fix this?

migrations-1     | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-1142396be754e21cc4a78e06971f59a7f8e93928.pack': Permission denied
migrations-1     | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-1142396be754e21cc4a78e06971f59a7f8e93928.idx': Permission denied
migrations-1     | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-54d2a9b6e4fa977aff5571d90743b6833b2ca907.idx': Permission denied
migrations-1     | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-54d2a9b6e4fa977aff5571d90743b6833b2ca907.pack': Permission denied
celery_worker-1  | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-1142396be754e21cc4a78e06971f59a7f8e93928.pack': Permission denied
web-1            | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-1142396be754e21cc4a78e06971f59a7f8e93928.pack': Permission denied
celery_worker-1  | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-1142396be754e21cc4a78e06971f59a7f8e93928.idx': Permission denied
web-1            | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-1142396be754e21cc4a78e06971f59a7f8e93928.idx': Permission denied
celery_worker-1  | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-54d2a9b6e4fa977aff5571d90743b6833b2ca907.idx': Permission denied
web-1            | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-54d2a9b6e4fa977aff5571d90743b6833b2ca907.idx': Permission denied
celery_worker-1  | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-54d2a9b6e4fa977aff5571d90743b6833b2ca907.pack': Permission denied
web-1            | chown: changing ownership of '/home/mediacms.io/mediacms/.git/objects/pack/pack-54d2a9b6e4fa977aff5571d90743b6833b2ca907.pack': Permission denied
tokenwizard commented 9 months ago

As mentioned in #894 (comment), this is just a message that can be ignored. You should be able to access http://localhost while the console is still printing these logs. However, if you want to remove these logs, one solution is to add the PGUSER variable in the docker-compose.yaml file：

db:
    image: postgres:15.2-alpine
    volumes:
      - ../postgres_data:/var/lib/postgresql/data/
    restart: always
    environment:
      POSTGRES_USER: mediacms
      POSTGRES_PASSWORD: mediacms
      POSTGRES_DB: mediacms
      PGUSER: mediacms # add here
      ...

Other solutions are mentioned here.

While I can confirm this does fix the root role errors cluttering up the console, unfortunately it does not fix the non-start issue. I have done a docker-composer up and let it sit for hours and the db_1 and redis_1 containers start up without error, but the web container never starts or is mentioned in the console. It just seems to hang there after starting redis_1 and is never accessible via the web interface.

vijaykumar1710 commented 9 months ago

As mentioned in #894 (comment), this is just a message that can be ignored. You should be able to access http://localhost while the console is still printing these logs. However, if you want to remove these logs, one solution is to add the PGUSER variable in the docker-compose.yaml file：

db:
    image: postgres:15.2-alpine
    volumes:
      - ../postgres_data:/var/lib/postgresql/data/
    restart: always
    environment:
      POSTGRES_USER: mediacms
      POSTGRES_PASSWORD: mediacms
      POSTGRES_DB: mediacms
      PGUSER: mediacms # add here
      ...

Other solutions are mentioned here.

While I can confirm this does fix the root role errors cluttering up the console, unfortunately it does not fix the non-start issue. I have done a docker-composer up and let it sit for hours and the db_1 and redis_1 containers start up without error, but the web container never starts or is mentioned in the console. It just seems to hang there after starting redis_1 and is never accessible via the web interface.

in my case i can access application on localhost but backend(api calls failing) seems to be not working by throwing errors. as api's are failing it is waste with frontend working.

tbmedia commented 9 months ago

I have the exact same problem on Docker for Windows. The web container simply doesn't want to run... which means this entire application doesn't work for me, and perhaps many others judging by this thread here.

Is there anything we are all collectively missing? I've tried the standard port 80, as well as alternative ports (xxxx:80). Docker Desktop is giving me the following error log:

(HTTP code 400) unexpected - failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./deploy/docker/entrypoint.sh": stat ./deploy/docker/entrypoint.sh: no such file or directory: unknown

joeyipjoeyip commented 8 months ago

Had the same problem. However it works when I switched to Rancher Desktop from Docker Desktop, see if it helps.

If Docker Desktop is required, I worked around it by manually running scripts (entrypoint.sh, prestart.sh, start.sh) in e.g. ./deploy/docker/prestart.sh manner. Running script by cd /deploy/docker first then e.g. ./prestart.sh causes some other error. May need to manually run the lines related to migration in prestart.sh for the first time. This workaround needs manually running scripts to start the web container everytime afterwards though.

MZDN commented 8 months ago

Had the same problem. However it works when I switched to Rancher Desktop from Docker Desktop, see if it helps.

If Docker Desktop is required, I worked around it by manually running scripts (entrypoint.sh, prestart.sh, start.sh) in e.g. ./deploy/docker/prestart.sh manner. Running script by cd /deploy/docker first then e.g. ./prestart.sh causes some other error. May need to manually run the lines related to migration in prestart.sh for the first time. This workaround needs manually running scripts to start the web container everytime afterwards though.

Thanks for this comment. Could you please give more details about the work around? In which container did you run the scripts? when running in Docker Desktop, following containers are appearing:

mediacms-celery_beat-1 (failing) mediacms-migrations-1 (failing) mediacms-db-1 (running)

joaogdfaero commented 8 months ago

I'm getting the same error. My Docker compose version never launchs. Any update?

tmyflyte commented 7 months ago

I tried to deploy a fresh version besides my old running instance (which mysteriously died with a 502 error after an update), but I'm getting the same problems as everyone else here. There's a problem apparently with the db, I can see the mediacms homepage anyhow but never a sign-in button and when I actually head to the sign-in page, all I see is a django error page :/

I'm getting this every time when trying to run:

WARN[0000] The "POSTGRES_DB" variable is not set. Defaulting to a blank string. WARN[0000] The "POSTGRES_USER" variable is not set. Defaulting to a blank string.

Even though my docker-compose.yaml has these set, as well as deploy/docker/local_settings.py

c0pper commented 5 months ago

I have the exact same problem on Docker for Windows. The web container simply doesn't want to run... which means this entire application doesn't work for me, and perhaps many others judging by this thread here.

Is there anything we are all collectively missing? I've tried the standard port 80, as well as alternative ports (xxxx:80). Docker Desktop is giving me the following error log:

(HTTP code 400) unexpected - failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "./deploy/docker/entrypoint.sh": stat ./deploy/docker/entrypoint.sh: no such file or directory: unknown

Same error here. Glad this seems to be a widespread issue with the docker compose deploy.

thau0x01 commented 3 weeks ago

Hi guys, I found out the problem. It seems to be some permission issues while running the containers on a OSX computer. The line 31 in deploy/docker/entrypoint.sh file tries to assert the correct file permissions for the www-data user. it works essentialy for all files copied into the folder during the docker image build process. However, since the project's root directory is mounted as volume in the same directory /home/mediacms.io/mediacms/, the .git folder is then placed under /home/mediacms.io/mediacms/.git with root permissions, causing the command find /home/mediacms.io/ ! \( -user www-data -group $TARGET_GID \) -exec chown www-data:$TARGET_GID {} + to fail. Once it fails, the migrations and other tasks the containers depends on also fail, causing the errors mentioned by @vijaykumar1710 and @johann-taberlet.

My solution was to modify thefind command above in the deploy/docker/entrypoint.sh file to this one: 

find /home/mediacms.io/ ! \( -path "*.git*" \) -and ! \( -user www-data -group $TARGET_GID \) -exec chown www-data:$TARGET_GID {} +

This command also ignores anything in the .git directory, avoiding the permission errors during the containers launch.

I've also made the PR #1052 to fix this issue.

smtsul commented 1 week ago

Hi guys, I think I fixed the problem for Windows. You need to open all the .sh files in the deploy/docker path with Notepad++ and convert them to Unix (LF) line endings.

mgogoulos commented 1 week ago

Hi all, thanks anyone that participated in this thread, I've merged the PRs that fix this and it should not be aproblem anymore, can you checkout the latest main branch and try it out?

Thanks a lot!