accept4: too many open files

[luckyraul]

http: Accept error: accept unix /run/docker/plugins/local-persist.sock: accept4: too many open files; retrying in 1s Apr 4 06:14:48 mon docker-volume-local-persist[14133]: 2016/04/04 06:14:48 http: Accept error: accept unix /run/docker/plugins/local-persist.sock: accept4: too many open files; retrying in 1s Apr 4 06:14:49 mon docker-volume-local-persist[14133]: 2016/04/04 06:14:49 http: Accept error: accept unix /run/docker/plugins/local-persist.sock: accept4: too many open files; retrying in 1s

[CWSpear]

Not really a lot of details to work with, but I found this on StackOverflow: http://stackoverflow.com/questions/880557/socket-accept-too-many-open-files

[luckyraul]

@CWSpear
This is not about system, this is all about your service.

Docker, for instance, has such code in default init.d script

ulimit -n 1048576
if [ "$BASH" ]; then
    ulimit -u 1048576
else
    ulimit -p 1048576
fi

[luckyraul]

or

LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity

in systemd

[CWSpear]

Again, you didn't really give me very much detail, so it's hard to tell.

What are the steps you're taking to produce this issue? Does it happen on plugin start? Does it happen when you try to start Docker containers? Does it happen when you try to create Volume with the plugin? What is the host OS where the plugin is running? Are you using any other plugins? What version of Docker are you using in the client? on the server?

Any other details you might think are relevant. It would help if it's reproducable to give me specific steps to reproduce it so I can reproduce it locally to try and debug.

[luckyraul]

docker 1.10, debian 8, kernel 4.4, no other plugins. This happens after few hours of normal working and can be fixed only by restarting service.

[CWSpear]

How many volumes are you using? Are you using Docker Compose? Could you share that config?

[luckyraul]

I created just small test stand. This server is a part of swarm cluster. At start it had only 1 volume via docker compose 1.6. Then my composer project moved to another server and I wanted to remove some used volumes and docker volume ls command freezed. Then I found such every second messages in logs.

[luckyraul]

Now It has only swarm agent and the same messages flow still is going

[rgamez]

I experienced similar errors using an older version of docker swarm which was leaking TCP sockets but note that I was NOT using local-persist, it was solved upgrading to the swarm 1.1.3 version.

[luckyraul]

Anyway I suggest to add parameters to systemd

[CWSpear]

What parameters?

[luckyraul]

LimitNOFILE=1048576 LimitNPROC=1048576 LimitCORE=infinity

[luckyraul]

My issue still present even with latest swarm