The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached.

[gregology]

I'm getting this error when I $ docker-compose up

birdsitelive | Unhandled exception. System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached.
birdsitelive |    at System.IO.FileSystemWatcher.StartRaisingEvents()
birdsitelive |    at System.IO.FileSystemWatcher.StartRaisingEventsIfNotDisposed()
birdsitelive |    at System.IO.FileSystemWatcher.set_EnableRaisingEvents(Boolean value)
birdsitelive |    at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
birdsitelive |    at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
birdsitelive |    at Microsoft.Extensions.FileProviders.PhysicalFileProvider.Watch(String filter)
birdsitelive |    at Microsoft.Extensions.Configuration.FileConfigurationProvider.<.ctor>b__1_0()
birdsitelive |    at Microsoft.Extensions.Primitives.ChangeToken.ChangeTokenRegistration`1..ctor(Func`1 changeTokenProducer, Action`1 changeTokenConsumer, TState state)
birdsitelive |    at Microsoft.Extensions.Primitives.ChangeToken.OnChange(Func`1 changeTokenProducer, Action changeTokenConsumer)
birdsitelive |    at Microsoft.Extensions.Configuration.FileConfigurationProvider..ctor(FileConfigurationSource source)
birdsitelive |    at Microsoft.Extensions.Configuration.Json.JsonConfigurationSource.Build(IConfigurationBuilder builder)
birdsitelive |    at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
birdsitelive |    at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
birdsitelive |    at Microsoft.Extensions.Hosting.HostBuilder.Build()
birdsitelive |    at BirdsiteLive.Program.Main(String[] args) in /src/BirdsiteLive/Program.cs:line 19
birdsitelive exited with code 139

I am using the comsumerkey and consumersecret from a Twitter standalone app I created.

this is my redacted docker-compose.yml

version: "3"

networks:
    birdsitelivenetwork:
        external: false

services:
    server:
        image: nicolasconstant/birdsitelive:latest
        restart: always
        container_name: birdsitelive
        environment:
            - Instance:Domain=birdsite.url
            - Instance:greg@email.com
            - Db:Type=postgres
            - Db:Host=db
            - Db:Name=birdsitelive
            - Db:User=birdsitelive
            - Db:Password=birdsitelive
            - Twitter:ConsumerKey=foo
            - Twitter:ConsumerSecret=bar
        networks:
            - birdsitelivenetwork
        ports:
            - "6000:80"
        depends_on:
            - db

    db:
        image: postgres:9.6
        restart: always
        environment:
            - POSTGRES_USER=birdsitelive
            - POSTGRES_PASSWORD=birdsitelive
            - POSTGRES_DB=birdsitelive
        networks:
            - birdsitelivenetwork
        volumes:
            - /home/user/birdsitelive/postgres:/var/lib/postgresql/data

any ideas for trouble shooting?

[NicolasConstant]

There is some solutions given on StackOverflow, maybe try adding those environment variables to see if it changes anything?

Is the hardware you're using exotic? (virtualized, etc?)

[jurgenhaas]

@NicolasConstant I'm having the same problem. The only env variables in the link above is

DOTNET_HOSTBUILDER__RELOADCONFIGONCHANGE=false

and I set that on the host and in the container, with no luck. I'm running on a root server with Ubunutu 22, so not really exitoc I guess.

[NicolasConstant]

Can you give me a bit more information so that I could try to reproduce on my side? What CPU/RAM/etc are you using? And also the exact version of docker / OS / etc.

[jurgenhaas]

Hmm, the problem started for me yesterday after an update of containerd.io - and today there was another update to containerd.io and the problem is now gone. It looks very much like being related.

[NicolasConstant]

Since it might be linked to 3rd party containizer software, I'm closing this. Please reply if the issue is still present.

[jurgenhaas]

This problem has not gone away. And I found a reproducible scenario and how to "fix" that. @NicolasConstant maybe worth re-opening the issue for that? At least more people could scratch their head around this problem and what we all could do about it.

Here are my observations: This always happens when containerd.io gets an update. During the update, that service gets restarted, but not the underlying docker service. That scenario then causes the issue described above.

To resolve this, you have to go through 3 steps:

• restart containerd again
• restart docker
• restart birdsite

I've seen this now a couple of times, and was able to resolve it this way each time.

Now, the originating problem seems to be the restart of containerd without also restarting docker. On the other hand, none of my other docker compose projects has any issues with that, it's only birdsite. And I wouldn't be surprised if it was somehow cause by the Microsoft libraries contained in that image, but I don't know anything about them.

[NicolasConstant]

Maybe it could be a good thing to also report this to containerd.io since it seems to be a bug on their side?

My docker image is made on the pattern provided by MS, but on next release I'll be using a more recent version of .NET, maybe we'll see some improvements. 🙂

[gregology]

+1 to reopening, the initial issue was not related to containerd.io. Sorry, I should have chirped in earlier

[vrutkovs]

echo fs.inotify.max_user_instances=524288 | sudo tee -a /etc/sysctl.conf && sudo sysctl -p

helped in my case (running BirdSiteLive in containerd using k3s)