Consider adding a recommendation to explicitly set an ulimit value in docker-compose.yml

[ashprice]

Using the default docker-compose.yml I would run into an issue where the database would fail to connect to MQ.

Setup would spawn two erl_child_setup processes which would each use 100% of a thread and kind of just... hang there.

Regular setup for mq would eventually continue (after about 20 minutes, with very slow progress of this thereafter), with the user 'sir' being created, and the correct vhost being created for the indexer, but prior to figuring out the issue I had a host of errors trying to run the regular setup commands for the indexer and amqp, because presumably other services had not completed whatever it was they needed to do, even after running for a significant period of time. To be clear, even after I had received the log messages about the vhost and amqp being set up correctly, and building the indexes etc, exactly according to the instructions, amqp setup would fail with a message saying that sir couldn't connect to the vhost.

I see there are potentially related issues (but I am not sure, as I am a bit of a neophyte), such as here, and there is a closed issue here.

I am not sure strictly which of the services I should set it for and if it harms the performance of the other services by setting it where I shouldn't, but setting universally

    ulimits:
      nofile: 65536

for all services in the docker-compose file fixes the issue.

I assume given I haven't seen this raised before in the issues here that this is either obvious to people who use this software regularly, or that the issues I've experienced are hardware specific, or that I did actually make some mistake with regard to the order of the instructions (I doubt this, as I went through the process multiple times...)

I'm not sure if the logs are preserved, but if I recreate the database at somepoint in the near future (a possibility) I will likely be able to recreate it and post them here.

I suppose it's not really required for this issue, but:

> git describe --always --broken --dirty --tags
v-2023-08-31-dirty
> docker-compose version --short
2.20.3
> sudo docker version -f 'Docker Client/Server: {{.Client.Version}}/{{.Server.Version}}'
Docker Client/Server: 24.0.5/24.0.5

where git indicates dirty because of docker-compose.yml having the modified ulimit value (and nothing else modified from the master branch).

[yvanzo]

Thank you for having collected those details. I have seen this issue happening in the past but I have not been able to reproduce it recently. From your second link, it seems that it depends on the host default settings.

It should be set for the service mq.

Can you please test on your host the pull request https://github.com/metabrainz/musicbrainz-docker/pull/258 I just opened?

git checkout -- docker-compose.yml # Remove your local changes to this file
git fetch origin
git checkout mq-ulimits
docker-compose up -d

[ashprice]

Sorry for the late reply.

This works fine. Running without the commit and with the defaults, the issue appears as expected. I get no log messages about the user sir or the vhost being readded - both are just there - so I am not sure how running docker-compose up -d after checking out the commit differs from initial setup, and if that would potentially matter, but I would guess not.

[yvanzo]

Sorry for the late reply too.

I have been able to test it on another host where the issue could be reproduced.

Now I merged the patch.

I get no log messages about the user sir or the vhost being readded

Actually the containers are marked as Recreated but it is quickly overridden with Started in the terminal output.