Problems running Swoole via a docker container in swarm mode

[weierophinney]

Please answer these questions before submitting your issue. Thanks!

1. What did you do? If possible, provide a recipe for reproducing the error.

I am project lead on zend-expressive-swoole, where we are providing a turn-key solution for deploying Expressive-based applications using Swoole.

I'm attempting to test how Swoole works in production, and specifically trying to deploy a Docker container based on php:7.2-cli in which I've built and enabled the Swoole extension:

 # DOCKER-VERSION        1.3.2

 FROM php:7.2-cli

 # System dependencies
 RUN apt-get update && \
   apt-get install -y git libbz2-dev libicu-dev libtidy-dev libxslt1-dev zlib1g-dev libc6-dev && \
   rm -rf /var/lib/apt/lists/*

 # Install Swoole
 RUN pecl install swoole-4.1.1 && docker-php-ext-enable swoole

 # Swoole configuration
 COPY swoole.ini /usr/local/etc/php/conf.d/000-swoole.ini

 # Install composer
 COPY getcomposer.sh /usr/local/bin/
 RUN /usr/local/bin/getcomposer.sh

 # Project files
 RUN mkdir -p /var/www/public
 COPY index.php /var/www/public/
 COPY entrypoint /usr/local/bin/

 EXPOSE 9000
 ENTRYPOINT ["entrypoint"]

The entrypoint is a PHP script that runs a Swoole HTTP Server.

2. What did you expect to see?

I expect to see Swoole running, and to be able to access the website via a browser.

3. What did you see instead?

When I use this container via docker compose on my local machiine, everything works fine. However, when I deploy it via swarm mode to a DigitalOcean droplet running Docker 18.01, the container constantly exits and restarts, with the following in the logs:

WARNING swShareMemory_mmap_create: mmap(260046944) failed. Error: Cannot allocate memory[12]
ERROR   calloc[1] failed

I've tried a number of different memory_limit settings, from disabling it (-1) to setting it to roughly half the available memory of the droplet (which reports 512M, so I set memory_limit to 256M), to setting it to a small value (64M). In all cases, I receive the above errors, but, as noted, only in swarm mode on the droplet, not when running docker compose using the same container locally.

4. What version of Swoole are you using (php --ri swoole)?

4.1.1

5. What is your machine environment used (including version of kernel & php & gcc) ?

The docker host reports: Linux 4.4.0-134-generic #160-Ubuntu SMP x86_64 x86_64 x86_64 GNU/Linux

The container is using the same kernel and architecture, and gcc version 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)

6. If you are using ssl, what is your openssl version?

Not applicable.

[twose]

Hi @weierophinney I found a possible reason, in reactor thread, The memory is allocated according to the max_connection parameter. When it is not set by the developer, the value of ulimit -n would be used. The value of ulimit -n in docker maybe too big, then the memory will be exceeded. You can set an appropriate max_conn num and the problem is solved.

$server->set(['max_conn' => 1000]);

[weierophinney]

I did a few experiments.

First, I created a new DO droplet, and discovered that my Swoole container would run fine in it!

However, once I deployed something non-trivial and started hitting it more often, I found I was getting errors of the following sort:

swServer_signal_hanlder: Fatal Error: manager process exit. status=0, signal=9.

At that point, I tried your suggestion above. I first fired up the container using /bin/bash as an entry point and ran ulimit -n to see what that returned; it was just over 1 million. Since I wasn't sure what a good number would be, I tried 1000 as you suggested.

The results: it is stable. In fact, I ran ab -k -c 1000 -t 10 {site} against it, and while it could not answer all the requests, the Swoole server never died or had errors!

I'm going to try upping the value to find a good threshold; in the meantime, thank you for the pointer, and the wonderful work on Swoole!

[twose]

yes, to find a good threshold, I just mention one example value here.