DocumentServer not running on Rocky 9 - RabbitMQ stuck starting

[nmaludy]

Do you want to request a feature or report a bug? Bug

What is the current behavior? Starting onlyoffice/documentserver container does not complete on Rocky Linux 9.3. It is stuck in service rabbitmq-server start

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. docker run onlyoffice/documentserver

What is the expected behavior? The container starts up without issue.

Did this work in previous versions of DocumentServer? Yes

DocumentServer Docker tag: 7.5.1.1

Host Operating System: Rocky Linux 9.3

I edited run-document-server.sh to include set -x so i can see where in the startup things were failing. Below are the logs

$ docker logs b7e50cfa4e77

+ umask 0022
+ trap clean_exit SIGTERM
+ shopt -s globstar
+ APP_DIR=/var/www/onlyoffice/documentserver
+ DATA_DIR=/var/www/onlyoffice/Data
+ PRIVATE_DATA_DIR=/var/www/onlyoffice/Data/.private
+ DS_RELEASE_DATE=/var/www/onlyoffice/Data/.private/ds_release_date
+ LOG_DIR=/var/log/onlyoffice
+ DS_LOG_DIR=/var/log/onlyoffice/documentserver
+ LIB_DIR=/var/lib/onlyoffice
+ DS_LIB_DIR=/var/lib/onlyoffice/documentserver
+ CONF_DIR=/etc/onlyoffice/documentserver
+ IS_UPGRADE=false
+ ONLYOFFICE_DATA_CONTAINER=true
+ ONLYOFFICE_DATA_CONTAINER_HOST=localhost
+ ONLYOFFICE_DATA_CONTAINER_PORT=80
++ sed -r 's/=([0-9]+)-([0-9]+)-([0-9]+) ([0-9:.+ ]+)/\1-\2-\3/'
++ stat -c=%y /var/www/onlyoffice/documentserver/server/DocService/docservice
+ RELEASE_DATE=2023-10-17
+ '[' -f /var/www/onlyoffice/Data/.private/ds_release_date ']'
++ head -n 1 /var/www/onlyoffice/Data/.private/ds_release_date
+ PREV_RELEASE_DATE=2023-10-17
+ '[' 2023-10-17 '!=' 2023-10-17 ']'
+ SSL_CERTIFICATES_DIR=/usr/share/ca-certificates/ds
+ mkdir -p /usr/share/ca-certificates/ds
+ [[ -d /var/www/onlyoffice/Data/certs ]]
+ '[' -e /var/www/onlyoffice/Data/certs/tls.crt ']'
+ cp -f /var/www/onlyoffice/Data/certs/dhparam.pem /var/www/onlyoffice/Data/certs/tls.crt /var/www/onlyoffice/Data/certs/tls.key /usr/share/ca-certificates/ds
+ chmod 644 /usr/share/ca-certificates/ds/tls.crt /usr/share/ca-certificates/ds/dhparam.pem
+ chmod 400 /usr/share/ca-certificates/ds/tls.key
+ [[ -z '' ]]
+ [[ -f /usr/share/ca-certificates/ds/onlyoffice.crt ]]
+ SSL_CERTIFICATE_PATH=/usr/share/ca-certificates/ds/tls.crt
+ [[ -z '' ]]
+ [[ -f /usr/share/ca-certificates/ds/onlyoffice.key ]]
+ SSL_KEY_PATH=/usr/share/ca-certificates/ds/tls.key
+ CA_CERTIFICATES_PATH=/usr/share/ca-certificates/ds/ca-certificates.pem
+ SSL_DHPARAM_PATH=/usr/share/ca-certificates/ds/dhparam.pem
+ SSL_VERIFY_CLIENT=off
+ USE_UNAUTHORIZED_STORAGE=false
+ ONLYOFFICE_HTTPS_HSTS_ENABLED=true
+ ONLYOFFICE_HTTPS_HSTS_MAXAGE=31536000
+ SYSCONF_TEMPLATES_DIR=/app/ds/setup/config
+ NGINX_CONFD_PATH=/etc/nginx/conf.d
+ NGINX_ONLYOFFICE_PATH=/etc/onlyoffice/documentserver/nginx
+ NGINX_ONLYOFFICE_CONF=/etc/onlyoffice/documentserver/nginx/ds.conf
+ NGINX_ONLYOFFICE_EXAMPLE_PATH=/etc/onlyoffice/documentserver-example/nginx
+ NGINX_ONLYOFFICE_EXAMPLE_CONF=/etc/onlyoffice/documentserver-example/nginx/includes/ds-example.conf
+ NGINX_CONFIG_PATH=/etc/nginx/nginx.conf
+ NGINX_WORKER_PROCESSES=1
++ ulimit -n
+ '[' 1073741816 -gt 1048576 ']'
+ NGINX_WORKER_CONNECTIONS=1048576
+ JWT_ENABLED=true
+ '[' true == true ']'
+ JWT_ENABLED=true
+ '[' -z xxx ']'
+ JWT_SECRET=xxx
+ JWT_HEADER=Authorization
+ JWT_IN_BODY=false
+ WOPI_ENABLED=false
+ ALLOW_META_IP_ADDRESS=false
+ ALLOW_PRIVATE_IP_ADDRESS=false
+ GENERATE_FONTS=true
+ [[ documentserver == \d\o\c\u\m\e\n\t\s\e\r\v\e\r ]]
+ REDIS_ENABLED=false
+ ONLYOFFICE_DEFAULT_CONFIG=/etc/onlyoffice/documentserver/local.json
+ ONLYOFFICE_LOG4JS_CONFIG=/etc/onlyoffice/documentserver/log4js/production.json
+ ONLYOFFICE_EXAMPLE_CONFIG=/etc/onlyoffice/documentserver-example/local.json
+ JSON_BIN=/var/www/onlyoffice/documentserver/npm/json
+ JSON='/var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json'
+ JSON_LOG='/var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/log4js/production.json'
+ JSON_EXAMPLE='/var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver-example/local.json'
+ LOCAL_SERVICES=()
+ PG_ROOT=/var/lib/postgresql
+ PG_NAME=main
+ PGDATA=/var/lib/postgresql/14/main
+ PG_NEW_CLUSTER=false
+ RABBITMQ_DATA=/var/lib/rabbitmq
+ REDIS_DATA=/var/lib/redis
+ '[' '' '!=' '' -a '' '!=' '' ']'
+ for i in converter docservice metrics
+ mkdir -p /var/log/onlyoffice/documentserver/converter
+ for i in converter docservice metrics
+ mkdir -p /var/log/onlyoffice/documentserver/docservice
+ for i in converter docservice metrics
+ mkdir -p /var/log/onlyoffice/documentserver/metrics
+ mkdir -p /var/log/onlyoffice/documentserver-example
+ for i in ${DS_LIB_DIR}/App_Data/cache/files ${DS_LIB_DIR}/App_Data/docbuilder ${DS_LIB_DIR}-example/files
+ mkdir -p /var/lib/onlyoffice/documentserver/App_Data/cache/files
+ for i in ${DS_LIB_DIR}/App_Data/cache/files ${DS_LIB_DIR}/App_Data/docbuilder ${DS_LIB_DIR}-example/files
+ mkdir -p /var/lib/onlyoffice/documentserver/App_Data/docbuilder
+ for i in ${DS_LIB_DIR}/App_Data/cache/files ${DS_LIB_DIR}/App_Data/docbuilder ${DS_LIB_DIR}-example/files
+ mkdir -p /var/lib/onlyoffice/documentserver-example/files
+ for i in ${LOG_DIR} ${LIB_DIR}
+ chown -R ds:ds /var/log/onlyoffice
+ chmod -R 755 /var/log/onlyoffice
+ for i in ${LOG_DIR} ${LIB_DIR}
+ chown -R ds:ds /var/lib/onlyoffice
+ chmod -R 755 /var/lib/onlyoffice
+ '[' localhost = localhost ']'
+ read_setting
+ deprecated_var POSTGRESQL_SERVER_HOST DB_HOST
+ [[ -n '' ]]
+ deprecated_var POSTGRESQL_SERVER_PORT DB_PORT
+ [[ -n '' ]]
+ deprecated_var POSTGRESQL_SERVER_DB_NAME DB_NAME
+ [[ -n '' ]]
+ deprecated_var POSTGRESQL_SERVER_USER DB_USER
+ [[ -n '' ]]
+ deprecated_var POSTGRESQL_SERVER_PASS DB_PWD
+ [[ -n '' ]]
+ deprecated_var RABBITMQ_SERVER_URL AMQP_URI
+ [[ -n '' ]]
+ deprecated_var AMQP_SERVER_URL AMQP_URI
+ [[ -n '' ]]
+ deprecated_var AMQP_SERVER_TYPE AMQP_TYPE
+ [[ -n '' ]]
+ METRICS_ENABLED=false
+ METRICS_HOST=localhost
+ METRICS_PORT=8125
+ METRICS_PREFIX=.ds
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json services.CoAuthoring.sql.dbHost
+ DB_HOST=localhost
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json services.CoAuthoring.sql.type
+ DB_TYPE=postgres
+ case $DB_TYPE in
+ DB_PORT=5432
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json services.CoAuthoring.sql.dbName
+ DB_NAME=onlyoffice
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json services.CoAuthoring.sql.dbUser
+ DB_USER=onlyoffice
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json services.CoAuthoring.sql.dbPass
+ DB_PWD=onlyoffice
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json rabbitmq.url
+ RABBITMQ_SERVER_URL=amqp://guest:guest@localhost
+ AMQP_URI=amqp://guest:guest@localhost
+ AMQP_TYPE=rabbitmq
+ parse_rabbitmq_url amqp://guest:guest@localhost
+ local amqp=amqp://guest:guest@localhost
++ grep ://
++ sed '-es,^\(.*://\).*,\1,g'
++ echo amqp://guest:guest@localhost
+ local proto=amqp://
++ echo guest:guest@localhost
+ local url=guest:guest@localhost
++ grep @
++ cut -d@ -f1
++ echo guest:guest@localhost
+ local userpass=guest:guest
++ grep :
++ cut -d: -f2
++ echo guest:guest
+ local pass=guest
+ local user
+ '[' -n guest ']'
++ grep :
++ cut -d: -f1
++ echo guest:guest
+ user=guest
++ cut -d/ -f1
++ echo localhost
+ local hostport=localhost
++ grep :
++ sed -r 's_^.*:+|/.*$__g'
++ echo localhost
+ local port=
+ local host
+ '[' -n '' ']'
+ host=localhost
+ port=5672
++ cut -d/ -f2-
++ grep /
++ echo guest:guest@localhost
+ local path=
+ AMQP_SERVER_PROTO=amqp
+ AMQP_SERVER_HOST=localhost
+ AMQP_SERVER_USER=guest
+ AMQP_SERVER_PASS=guest
+ AMQP_SERVER_PORT=5672
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json services.CoAuthoring.redis.host
+ REDIS_SERVER_HOST=
+ REDIS_SERVER_PORT=6379
++ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/log4js/production.json categories.default.level
+ DS_LOG_LEVEL=WARN
+ '[' false = true ']'
+ update_welcome_page
+ WELCOME_PAGE=/var/www/onlyoffice/documentserver-example/welcome/docker.html
+ [[ -e /var/www/onlyoffice/documentserver-example/welcome/docker.html ]]
+++ cat /proc/1/cpuset
++ basename /
+ DOCKER_CONTAINER_ID=/
+ ((  1 < 12  ))
++ hostname
+ DOCKER_CONTAINER_ID=b7e50cfa4e77
+ ((  12 >= 12  ))
++ command -v docker
+ [[ -x '' ]]
+ sed 's/$(sudo docker ps -q)/b7e50cfa4e77/' -i /var/www/onlyoffice/documentserver-example/welcome/docker.html
++ sed 's/$(sudo docker ps -q)/b7e50cfa4e77/'
++ echo
+ JWT_MESSAGE=
+ update_log_settings
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/log4js/production.json -I -e 'this.categories.default.level = '\''WARN'\'''
+ update_ds_settings
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.enable.browser = true'
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.enable.request.inbox = true'
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.enable.request.outbox = true'
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.secret.inbox.string = '\''xxx'\'''
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.secret.outbox.string = '\''xxx'\'''
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.secret.session.string = '\''xxx'\'''
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.inbox.header = '\''Authorization'\'''
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.outbox.header = '\''Authorization'\'''
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.inbox.inBody = false'
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver/local.json -I -e 'this.services.CoAuthoring.token.outbox.inBody = false'
+ '[' -f /etc/onlyoffice/documentserver-example/local.json ']'
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver-example/local.json -I -e 'this.server.token.enable = true'
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver-example/local.json -I -e 'this.server.token.secret = '\''xxx'\'''
+ /var/www/onlyoffice/documentserver/npm/json -q -f /etc/onlyoffice/documentserver-example/local.json -I -e 'this.server.token.authorizationHeader = '\''Authorization'\'''
+ '[' false == true ']'
+ '[' false == true ']'
+ '[' false = true ']'
+ '[' false = true ']'
+ '[' localhost '!=' localhost ']'
+ chown -R postgres:postgres /var/lib/postgresql
+ chmod -R 700 /var/lib/postgresql
+ '[' '!' -d /var/lib/postgresql/14/main ']'
+ LOCAL_SERVICES+=("postgresql")
+ '[' localhost '!=' localhost ']'
+ chown -R rabbitmq:rabbitmq /var/lib/rabbitmq
+ chmod -R go=rX,u=rwX /var/lib/rabbitmq
+ '[' -f /var/lib/rabbitmq/.erlang.cookie ']'
+ chmod 400 /var/lib/rabbitmq/.erlang.cookie
+ LOCAL_SERVICES+=("rabbitmq-server")
+ rm -rf /var/run/rabbitmq
+ '[' false = true ']'
+ find /etc/onlyoffice '!' -path '*logrotate*' -exec chown ds:ds '{}' ';'
+ for i in ${LOCAL_SERVICES[@]}
+ service postgresql start
 * Starting PostgreSQL 14 database server                                [ OK ] 
+ for i in ${LOCAL_SERVICES[@]}
+ service rabbitmq-server start

[nmaludy]

Some more debugging info from inside the container

# service rabbitmq-server status
Error: unable to perform an operation on node 'rabbit@b7e50cfa4e77'. Please see diagnostics information and suggestions below.

Most common reasons for this are:

 * Target node is unreachable (e.g. due to hostname resolution, TCP connection or firewall issues)
 * CLI tool fails to authenticate with the server (e.g. due to CLI tool's Erlang cookie not matching that of the server)
 * Target node is not running

In addition to the diagnostics info below:

 * See the CLI, clustering and networking guides on https://rabbitmq.com/documentation.html to learn more
 * Consult server logs on node rabbit@b7e50cfa4e77
 * If target node is configured to use long node names, don't forget to use --longnames with CLI tools

DIAGNOSTICS
===========

attempted to contact: [rabbit@b7e50cfa4e77]

rabbit@b7e50cfa4e77:
  * connected to epmd (port 4369) on b7e50cfa4e77
  * epmd reports: node 'rabbit' not running at all
                  other nodes on b7e50cfa4e77: ['rabbitmqcli-305-rabbit']
  * suggestion: start the node

Current node details:
 * node name: 'rabbitmqcli-655-rabbit@b7e50cfa4e77'
 * effective user's home directory: /var/lib/rabbitmq
 * Erlang cookie hash: xxx

[igwyd]

Hello @nmaludy, i checket on my stand, but got no errors. Do you comply with the Software Requirements? Please note that there must be 4GB of RAM.

[nmaludy]

@igwyd Yes, I comply with all of those requirements.

I switched to the Snap package and it works just fine.

[igwyd]

Ok, then we close this issue. Feel free to comment or reopen it if you got further questions.