The configured user limit (128) on the number of inotify instances has been reached...

[ScruffyDerHausmeister]

I've got an error when opening a repository on linux (Zorin OS / Ubuntu).

Crash::: System.IO.IOException: The configured user limit (128) on the number of inotify instances has been reached, or the per-process limit on the number of open file descriptors has been reached. crash_2024-08-25_20-48-56.log

[gadfly3173]

You can add these to /etc/sysctl.conf .

fs.inotify.max_user_instances=1024
fs.inotify.max_user_watches=1048576

This error occurs when listening for a large number of files on Linux. Edit: To solve this problem from the software level, it may need to catch the IOException thrown when starting the Watcher and set the Watcher to polling mode.

[ScruffyDerHausmeister]

It'd be nice to find a solution without changing systemfiles which never were needed to be changed.

[love-linger]

Using lsof -c sourcegit command to show the file descriptors opened by SourceGit on Ubuntu 20.04 (WSL 2).

• After SourceGit started (Without any repository opened) empty.txt
• After a repository was opened in SourceGit open_sourcegit_repo.txt

[!NOTE] When a repository is opened in SourceGit, we will create two FileSystemWatcher - one for repository's git dir and another for repository's worktree.

The main difference:

[love-linger]

Another test

• Start without repository opened open_0_repo.txt
• Start with 1 repository opened open_1_repo.txt
• Start with 2 repositories opened open_2_repo.txt

It seems like that when we create a new FileSystemWatcher, a a_inode FD will be opened.

[love-linger]

According to the test results, the FDs did not increase beyond the expected situation.

[gadfly3173]

According to the test results, the FDs did not increase beyond the expected situation.

The upper limit that is usually reached is fs.inotify.max_user_instances. When I opened 4 IDEAs, 1 Rider, 2 Tomcats, and 1 SourceGit (opened 6 repositories), I could see that the number of instances reached 105. Opening another SourceGit will bring the number to 117.

   INOTIFY   INSTANCES
   WATCHES      PER   
    COUNT     PROCESS   PID USER         COMMAND
------------------------------------------------------------
   15703        13       40694 homolo      /opt/sourcegit/sourcegit
   14907         1        3981 homolo      /snap/intellij-idea-ultimate/524/bin/fsnotifier
    3414         1      140759 homolo      /snap/rider/489/bin/fsnotifier
      60         3       98184 homolo      /usr/libexec/dde-file-manager -d
      48         3       40197 homolo      /usr/bin/dde-select-dialog-x11
      21         3        1172 homolo      /usr/bin/dde-desktop
      19         4       22666 homolo      /opt/google/chrome/chrome

========ignore=============

INotify instances per user (e.g. limits specified by fs.inotify.max_user_instances): 

INSTANCES    USER
-----------  ------------------
105          homolo

It can be observed that IDEA only creates two instances. I think SourceGit can also consider reducing the number of inotify instances created.

   14907         1        3981 homolo      /snap/intellij-idea-ultimate/524/bin/fsnotifier
       9         1        3848 homolo      /snap/intellij-idea-ultimate/524/bin/idea

Edit: the script is https://github.com/fatso83/dotfiles/blob/master/utils/scripts/inotify-consumers

[love-linger]

https://github.com/user-attachments/assets/8a789bea-68a0-4d43-bbfa-48d6dbfc0554

@gadfly3173 I've test with this script https://github.com/fatso83/dotfiles/blob/master/utils/scripts/inotify-consumers

It shows when a repository is opened, 2 instances (needed by 2 FileSystemWatcher) are created.

Could you help me to test that what's the result when multiple instance of IDEA opened with different project? (We need to listen different paths not just one project)

[gadfly3173]

Could you help me to test that what's the result when multiple instance of IDEA opened with different project? (We need to listen different paths not just one project)您能否帮我测试一下，当使用不同项目打开多个 IDEA 实例时，结果是什么？（我们需要倾听不同的路径，而不仅仅是一个项目）

The previously mentioned IDEA creates only two instances is the situation when four different projects are opened.

   14907         1        3981 homolo      /snap/intellij-idea-ultimate/524/bin/fsnotifier
       9         1        3848 homolo      /snap/intellij-idea-ultimate/524/bin/idea

reduce to 2 project:

   12190         1        3981 homolo      /snap/intellij-idea-ultimate/524/bin/fsnotifier
       9         1        3848 homolo      /snap/intellij-idea-ultimate/524/bin/idea

Rider with SourceGit repository:

    3418         1      160173 homolo      /snap/rider/489/bin/fsnotifier
       9         1      160024 homolo      /snap/rider/489/bin/rider

[aikawayataro]

I think we can use single instance of inotify subsystem on Linux if we use P/Invoke and call inotify_add_watch with single subsystem This will not reduce the number of watches, though

EDIT: Or use fanotify which should work better for Source Git case

[gadfly3173]

Here is Gitkraken with 6 opened repositories. And I think this is a dotnet-runtime issue: https://github.com/dotnet/runtime/issues/62869

    2289         3      168727 homolo      /persistent/data/Gitkraken/gitkraken --type=renderer --crashpad-handler-pid=1686
       6         4      168590 homolo      /persistent/data/Gitkraken/gitkraken
       2         1      168648 homolo      /persistent/data/Gitkraken/gitkraken --type=utility --utility-sub-type=network.m

[love-linger]

I took a look at how to listen for filesystem changes in Java and found that starting Java 7, it provides 'java.nio.file.WatchService' and it allows register multiple paths. For example:

WatchService watchService = FileSystems.getDefault().newWatchService();
Path path1 = Paths.get("path1");
path1.register(watchService, StandardWatchEventKinds.ENTRY_CREATE);
Path path2 = Paths.get("path2");
path2.register(watchService, StandardWatchEventKinds.ENTRY_CREATE);

But in C#, the FileSystemWatcher.Path is a simple string. We need to create multiple instance of FileSystemWatcher to listen multiple paths.

[gadfly3173]

After observing the behavior of Gitkraken, I found that it would prompt File watching failed to start for this repository.. Then the repository would be opened normally, but the file status would not be automatically updated until I switched to this tab again.

[love-linger]

I've pushed a commit that deals with this in the way Gitkraken does and these two method still works without a Watcher instance:

public void MarkBranchesDirtyManually()
{
    if (_watcher == null)
    {
        Task.Run(() =>
        {
            RefreshBranches();
            RefreshCommits();
        });

        Task.Run(RefreshWorkingCopyChanges);
        Task.Run(RefreshWorktrees);
    }                
    else
    {
        _watcher.MarkBranchDirtyManually();
    }
}

public void MarkWorkingCopyDirtyManually()
{
    if (_watcher == null)
        Task.Run(RefreshWorkingCopyChanges);
    else
        _watcher.MarkWorkingCopyDirtyManually();
}