Very slow with many files on BindFS (FUSE filesystem)

[1-eric]

When KaraDAV has to serve files stored on a BindFS mountpoint, calculating size and showing the file list is very slow.

After clicking on the directory, it took 8 minutes before the file list is shown. This is a 200 MB directory with 400 files. Showing directories with more files will fail because of time-out. During this process, the bindfs task is using 80+% of the CPU.

BindFS (or FUSE filesystems) may be the cause of this, but I need it to map user, group and permissions for the files and directories.

This issue is partly related to issue #27, but the problem I have not only happens right after login (when the total size is calculated), but also after clicking on the directory.

If I remember correctly, Nextcloud had a cronjob. Maybe this cronjob is reading the directory once and then store all information in the database. Every time the directory is accessed, the database will be used.

[bohwaz]

Yup this is related to bindFS I think.You should disable quotas.

[1-eric]

To speed up directory listings, I've made the getDirectoryMTime function return 0:

static public function getDirectoryMTime(string $path): int
    {
        return 0;

This may break other things, but at least it works for the a mobile gallery app that must connect to an Owncloud/Nextcloud storage. This app only needs read-access, for writing and sync I use other apps and they are compatible with regular WebDAV or sftp.

[bohwaz]

Ah I forgot about that, thank you.

Yes this won't break things, it's just that the last modification date for a directory will not be accurate in OC/NC clients that's all.

I have added a new config constant (in release 0.3.9) to disable this and other slow operations (such as finding out the size of a directory), you can add this to your config.local.php:

const DISABLE_SLOW_OPERATIONS = true;

You should also disable quotas if your filesystem is slow.

This is because KaraDAV doesn't have any database storage of the files, it is just speaking directly to the filesystem, so unlike NextCloud, it cannot know the size of files you have, it has to ask the filesystem, and count every file in every subdirectory, and this might be slow with a large number of files.