Zammad not accesible, keeps saying waiting for init container to finish install or update...

[distributev]

Infos

• Docker version:

Docker version 24.0.6, build ed223bc

• Docker-compose version:

Docker Compose version v2.11.1

• Operating system (Docker host):

Debian 11

Expected behavior

Zammad should work when correct configuration is provided.

Actual behavior

Zammad is not accesible, the command docker logs zammad-railsserver keeps saying

waiting for init container to finish install or update...

zammad-ngix seems to have finished since it does not appears anymore in the output of docker ps

Most likely connection configuration is fine since all the other containers are listed in the docker ps including zammad-nginx, zammad-railserver,etc and they are not restarted (no errors)

I checked the logs for each container individually and I do not see any error like Could not connect to database or similar.

the command docker ps shows

e37924b5e3c9   postgres:15.3-alpine                     "/usr/local/bin/back…"   11 minutes ago   Up 11 minutes          5432/tcp                                                                   zammad-backup
aa1cba2f7519   ghcr.io/zammad/zammad:6.0.0-119          "/docker-entrypoint.…"   11 minutes ago   Up 11 minutes                                                                                     zammad-nginx
b104f4dd3e4f   ghcr.io/zammad/zammad:6.0.0-119          "/docker-entrypoint.…"   11 minutes ago   Up 11 minutes                                                                                     zammad-railsserver
9cea1e441ed5   redis:7.0.5-alpine                       "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes          6379/tcp                                                                   zammad-redis
6ea4626ed403   postgres:15.3-alpine                     "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes          5432/tcp                                                                   zammad-db
460a33c6da9c   bitnami/elasticsearch:8.8.0              "/opt/bitnami/script…"   11 minutes ago   Up 11 minutes          9200/tcp, 9300/tcp                                                         zammad-elasticsearch
e0319493a75a   memcached:1.6.20-alpine                  "docker-entrypoint.s…"   11 minutes ago   Up 11 minutes          11211/tcp                                                                  zammad-memcached
40e196ac45ec   ghcr.io/zammad/zammad:6.0.0-119          "/docker-entrypoint.…"   11 minutes ago   Up 11 minutes                                                                                     zammad-websocket
7dc86e8edbcc   ghcr.io/zammad/zammad:6.0.0-119          "/docker-entrypoint.…"   11 minutes ago   Up 11 minutes                                                                                     zammad-scheduler

docker logs zammad-elasticsearch shows

[2023-11-02T12:57:27,834][INFO ][o.e.n.Node               ] [460a33c6da9c] started {460a33c6da9c}{QWoVUbnvTuy6zR92bysbgQ}{a7H4yEhLRdyCYyjr5vHFIQ}{460a33c6da9c}{172.18.0.35}{172.18.0.35:9300}{cdfhilmrstw}{8.8.0}{xpack.installed=true}
[2023-11-02T12:57:27,842][INFO ][o.e.c.f.AbstractFileWatchingService] [460a33c6da9c] file settings service up and running [tid=34]
{"@timestamp":"2023-11-02T12:57:27.916Z", "log.level": "WARN",  "data_stream.dataset":"deprecation.elasticsearch","data_stream.namespace":"default","data_stream.type":"logs","elasticsearch.event.category":"parsing","event.code":"attachment-remove-binary","message":"The default [remove_binary] value of 'false' is deprecated and will be set to 'true' in a future release. Set [remove_binary] explicitly to 'true' or 'false' to ensure no behavior change." , "ecs.version": "1.2.0","service.name":"ES_ECS","event.dataset":"deprecation.elasticsearch","process.thread.name":"elasticsearch[460a33c6da9c][clusterApplierService#updateTask][T#1]","log.logger":"org.elasticsearch.deprecation.ingest.attachment.AttachmentProcessor","elasticsearch.cluster.uuid":"O_HXLGuVQkirfCBGwG42Tg","elasticsearch.node.id":"QWoVUbnvTuy6zR92bysbgQ","elasticsearch.node.name":"460a33c6da9c","elasticsearch.cluster.name":"elasticsearch"}
[2023-11-02T12:57:28,117][INFO ][o.e.l.ClusterStateLicenseService] [460a33c6da9c] license [4d9658aa-1d4f-41fb-a97f-667cdedc3d7e] mode [basic] - valid
[2023-11-02T12:57:28,122][INFO ][o.e.g.GatewayService     ] [460a33c6da9c] recovered [14] indices into cluster_state
[2023-11-02T12:57:28,216][INFO ][o.e.h.n.s.HealthNodeTaskExecutor] [460a33c6da9c] Node [{460a33c6da9c}{QWoVUbnvTuy6zR92bysbgQ}] is selected as the current health node.
[2023-11-02T12:57:29,296][INFO ][o.e.c.r.a.AllocationService] [460a33c6da9c] current.health="YELLOW" message="Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[zammad_production_cti_log][0], [zammad_production_chat_session][0]]])." previous.health="RED" reason="shards started [[zammad_production_cti_log][0], [zammad_production_chat_session][0]]"

docker logs zammad-nginx shows

starting nginx...
2023/11/02 12:57:48 [error] 16#16: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.20, server: _, request: "GET / HTTP/1.1", upstream: "http://172.18.0.38:3000/", host: "support.sourcekraft.net"
172.18.0.20 - - [02/Nov/2023:12:57:48 +0000] "GET / HTTP/1.1" 502 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36 OPR/103.0.0.0"
2023/11/02 12:57:49 [error] 16#16: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.20, server: _, request: "GET / HTTP/1.1", upstream: "http://172.18.0.38:3000/", host: "support.sourcekraft.net"
172.18.0.20 - - [02/Nov/2023:12:57:49 +0000] "GET / HTTP/1.1" 502 552 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36 OPR/103.0.0.0"

docker logs zammad-init shows

I, [2023-11-02T12:57:12.279244 #20]  INFO -- : ActionCable is using the redis instance at redis://zammad-redis:6379.
I, [2023-11-02T12:57:12.306972#20-5380]  INFO -- : Using memcached as Rails cache store.
I, [2023-11-02T12:57:12.307129#20-5380]  INFO -- : Using the Redis back end for Zammad's web socket session store.
W, [2023-11-02T12:57:12.642137#20-5380]  WARN -- : Setting "timezone_default" is empty. Using UTC instead. Please set system timezone.
I, [2023-11-02T12:57:15.640329#20-5380]  INFO -- : Setting.set('models_searchable', ["Chat::Session", "KnowledgeBase::Answer::Translation", "Organization", "Ticket", "User"])
I, [2023-11-02T12:57:17.040405#20-5380]  INFO -- : Setting.set('es_url', "http://zammad-elasticsearch:9200")
I, [2023-11-02T12:57:20.092927 #22]  INFO -- : ActionCable is using the redis instance at redis://zammad-redis:6379.
I, [2023-11-02T12:57:20.120803#22-5380]  INFO -- : Using memcached as Rails cache store.
I, [2023-11-02T12:57:20.121092#22-5380]  INFO -- : Using the Redis back end for Zammad's web socket session store.
W, [2023-11-02T12:57:20.326107#22-5380]  WARN -- : Setting "timezone_default" is empty. Using UTC instead. Please set system timezone.
I, [2023-11-02T12:57:23.053211#22-5380]  INFO -- : Setting.set('models_searchable', ["Chat::Session", "KnowledgeBase::Answer::Translation", "Organization", "Ticket", "User"])
I, [2023-11-02T12:57:24.124080#22-5380]  INFO -- : Setting.set('es_index', "zammad")
zammad railsserver waiting for elasticsearch server to be ready...

docker logs zammad-railsserver shows

waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...
waiting for init container to finish install or update...

Steps to reproduce the behavior

*

[mgruner]

It seems that the zammad-init container is not able to create a test connection to the Elasticsearch service, and therefore does not finish. This blocks subsequents containers.

I am not sure what could cause this, maybe resource or other environment issues? Please head over to https://community.zammad.org/ to ask for help with your set-up.

I'll close the current issue for now, as it is not about a (reproducible) issue in the Zammad software/stack. Please feel free to provide more information in future.

[distributev]

I believe the container logs could be improved in this situation - nowhere in the container logs I could see a good hint like ERROR or FATAL with details - zammad-init container even finished without giving an error code (I have it configured to restart if error and it did not) instead of showing 100 times 'waiting for init container to finish install or update...' without checking that init container already finished (without success) maybe after it says 5 or 10 times times 'waiting for init container to finish install or update...' it could say (after 10 iterations) ERROR and it is probably because .... and solutions could be 1. possible solution 1 if that 2. possible solution 2 if the other, etc.

[distributev]

I checked the elasticsearch logs line by line and besides, the last line which gives a hint that there could be an issue, there is not other line which suggests anything related with a problem but it doesn't say also anything like success - so no information.

Last few lines from elasticsearch suggest something is not well but it says about shards etc - not related with memory or resources.

[2023-11-02T12:57:28,216][INFO ][o.e.h.n.s.HealthNodeTaskExecutor] [460a33c6da9c] Node [{460a33c6da9c}{QWoVUbnvTuy6zR92bysbgQ}] is selected as the current health node.
[2023-11-02T12:57:29,296][INFO ][o.e.c.r.a.AllocationService] [460a33c6da9c] current.health="YELLOW" message="**_Cluster health status changed from [RED] to [YELLOW]_** (reason: [shards started [[zammad_production_cti_log][0], [zammad_production_chat_session][0]]])." previous.health="RED" reason="shards started [[zammad_production_cti_log][0], [zammad_production_chat_session][0]]"

[mgruner]

Thanks for the updates. I agree that the logs can be improved in future. However, there is still no reproducible error in the Zammad stack at the moment, and therefore I'll close this issue for now. Please feel free to provide more information.

[distributev]

zammad-init finished and shut down itself since 10 minutes, zammad-init logs show that it did work and zammad-init logs say nothing about any possible issue however zammad-railsserver and other containers keep displaying in an infinite loop "waiting for init container to finish install or update..." + the app is not accessible and there are no other logs which indicate any possible issue.

Is this the expected behavior?

I believe this is a bug, I'm playing all day long with lots of docker applications. I'm going to try chatwood hoping that their docker setup is going to be straightforward.

[MrGeneration]

Please consult the community as requested several times.

[distributev]

I already did and found some entries related with resources which could apply, tried few things and nothing worked - could not get past the infinite loop "waiting for init container to finish install or update..." - nevertheless the docker container logs could be improved to be (much) more specific and this is the reason I believe this is a bug.

[distributev]

Thank you for your efforts to build an open source support software.

When I tried v5 I was able to get it working using docker-compose, I'm not sure what changed in v6 that suddenly I'm not able to get it working. I even tried to inspire from the v5 docker-compose setup and to adapt it to v6, unfortunately without any luck.

[MrGeneration]

Sorry this is not helping anyone. I'm limitting this issue due to off topic and repeated messages to topics that should belong in our community forum.

6.0 is outdated, use 6.1.

[mgruner]

@distributev if you have any pointers to technical for us, please let us know. The init container writes a Zammad-ready file that the other containers look for to determine that the init container finished. You may want to check if that file is present.