deploy: Create and start services task always fails

[metalefty]

After https://github.com/AlmaLinux/albs-web-server/pull/792, there is a problem.

albs_test_db_1 container fails to start due to already used port 5432/tcp.

Error response from daemon: driver failed programming external connectivity on endpoint albs_test_db_1 (28412eeecc3feff17618ff42efb5a62eac6e673542f5e0e986f7b68ceefa09fd): Bind for 0.0.0.0:5432 failed: port is already allocated

MSG:

non-zero return code

NO MORE HOSTS LEFT ******************************************************************************************************************

PLAY RECAP **************************************************************************************************************************
albs_on_one_vm             : ok=27   changed=10   unreachable=0    failed=1    skipped=7    rescued=0    ignored=0

test_db service added by https://github.com/AlmaLinux/albs-web-server/pull/792 listens 5432/tcp despite the port is already used by db service.

• https://github.com/AlmaLinux/albs-web-server/blob/master/docker-compose.yml#L5-L10
• https://github.com/AlmaLinux/albs-web-server/blob/master/docker-compose.yml#L23-L38

[metalefty]

@isudak Ansible log says, albs_test_db_1 container is started before albs_db_1 container starts. albs_db_1 container cannot be started because the same port is already used by albs_test_db_1.

Full ansible log is here: ansible.log

time="2024-05-23T05:37:18Z" level=warning msg="/home/almalinux/albs/albs-web-server/docker-compose.yml: `version` is obsolete"
 Container albs_db_1  Recreate
 Container rabbitmq-ts  Recreate
 Container albs_sign_file_1  Recreate
 Container albs_task_queue_sign_1  Recreate
 Container albs_web_server_tests_1  Recreate
 Container albs_immudb_1  Recreate
 Container albs_redis_1  Recreate
 Container albs_pulp_1  Recreate
 Container albs_mosquitto_1  Recreate
 Container albs_test_db_1  Recreate
 Container albs_task_queue_product_modify_1  Recreate
 Container albs_sign_file_1  Recreated
 Container albs_db_1  Recreated
 Container albs_task_queue_sign_1  Recreated
 Container albs_mosquitto_1  Recreated
 Container albs_gitea_listener_1  Recreate
 Container rabbitmq-ts  Recreated
 Container albs_alts-celery_1  Recreate
 Container albs_task_queue_product_modify_1  Recreated
 Container albs_redis_1  Recreated
 Container albs_git_cacher_1  Recreate
 Container albs_web_server_tests_1  Recreated
 Container albs_test_db_1  Recreated
 Container albs_immudb_1  Recreated
 Container albs_pulp_1  Recreated
 Container albs_task_queue_errata_1  Recreate
 Container albs_task_queue_1  Recreate
 Container albs_task_queue_tests_1  Recreate
 Container albs_web_server_1  Recreate
 Container albs_task_queue_releases_1  Recreate
 Container albs_task_queue_builds_1  Recreate
 Container albs_gitea_listener_1  Recreated
 Container albs_alts-celery_1  Recreated
 Container albs_git_cacher_1  Recreated
 Container albs_web_server_1  Recreated
 Container albs_build_node_1  Recreate
 Container albs_frontend_1  Recreate
 Container albs_tests_cacher_1  Recreate
 Container albs_sign_node_1  Recreate
 Container albs_alts-scheduler_1  Recreate
 Container albs_task_queue_1  Recreated
 Container albs_task_queue_releases_1  Recreated
 Container albs_task_queue_tests_1  Recreated
 Container albs_task_queue_builds_1  Recreated
 Container albs_task_queue_errata_1  Recreated
 Container albs_sign_node_1  Recreated
 Container albs_tests_cacher_1  Recreated
 Container albs_alts-scheduler_1  Recreated
 Container albs_frontend_1  Recreated
 Container albs_nginx_1  Recreate
 Container albs_build_node_1  Recreated
 Container albs_nginx_1  Recreated
 Container albs_mosquitto_1  Starting
 Container albs_db_1  Starting
 Container albs_pulp_1  Starting
 Container albs_test_db_1  Starting
 Container albs_sign_file_1  Starting
 Container albs_web_server_tests_1  Starting
 Container rabbitmq-ts  Starting
 Container albs_immudb_1  Starting
 Container albs_redis_1  Starting
 Container albs_task_queue_product_modify_1  Starting
 Container albs_task_queue_sign_1  Starting
 Container albs_mosquitto_1  Started
 Container albs_mosquitto_1  Started
 Container albs_gitea_listener_1  Starting
 Container rabbitmq-ts  Started
 Container albs_alts-celery_1  Starting
 Container albs_task_queue_sign_1  Started
 Container albs_test_db_1  Started
 Container albs_task_queue_product_modify_1  Started
 Container albs_web_server_tests_1  Started
 Container albs_immudb_1  Started
 Container albs_redis_1  Started
 Container albs_sign_file_1  Started
 Container albs_pulp_1  Started
 Container albs_gitea_listener_1  Started
 Container albs_alts-celery_1  Started
Error response from daemon: driver failed programming external connectivity on endpoint albs_db_1 (62970255b998ea4128b98da92d188af912817f7f3b0f072c1c518ca14e392016): Bind for 0.0.0.0:5432 failed: port is already allocated

[isudak]

Starting every service found in docker-compose.yml doesn't look right. Maybe all images should be created, but only albs_web_server_1 has to be started?

[javihernandez]

Starting every service found in docker-compose.yml doesn't look right. Maybe all images should be created, but only albs_web_server_1 has to be started?

docker compose up will bring everything on docker-compose.yml up. In my case, I rarely do that when dealing with my dev env, and I don't like the idea of albs-deploy bringing up every service, but never wrote an issue or talked to anybody about this (shame on me). I'd say that we should only bring up the minimal viable system, as an example:

• web_server
• frontend
• build_node
• task_queue
• task_queue_builds
• git_cacher
• nginx

Then, ensure that we have a good documentation about it in albs-deploy or in our wiki. As for db, redis, etc, they will be started as they are dependencies of some of the services above. I believe that the machinery for that is already in albs-deploy, but is not used by default. What do you think?

[metalefty]

Thanks, then I think excluded_containers should be filled by default to start all necessary containers out-of-box.

[metalefty]

It might be better making a list of necessary containers rather than a list of containers to exclude.

[javihernandez]

It might be better making a list of necessary containers rather than a list of containers to exclude.

The logic used around excluded_containers can be found here. It looks solid to me, but I don't have a strong opinion on whether we should change it or not.