Filescan on users with large Groupfolder freezes NC

[kowjens]

Steps to reproduce

Use Groupfolders with some GB data in it shared to some users run filescan:all

Expected behaviour

just scan users files and than the group folder once instead scanning the whole group folder once per user

Actual behaviour

-> scan on every user who see groupfolder (even if only empty folders are visible to this user via ACL) leads to full scan of the files in groupfolder -> if groupfolder is e.g. 100GB and you have 30 users it scans 30x100GB and takes hours and hours for handmade scans and seems to make problems with cron leading to gateway timeouts and such failures. this makes the whole cloud not useable

Server configuration detail

Operating system: Linux 5.4.0-65-generic nextcloud/server#73-Ubuntu SMP Mon Jan 18 17:25:17 UTC 2021 x86_64

Webserver: nginx/1.19.6 (fpm-fcgi)

Database: mysql 10.2.14

PHP version:

7.4.15 Modules loaded: Core, date, libxml, openssl, pcre, sqlite3, zlib, ctype, curl, dom, fileinfo, filter, ftp, hash, iconv, json, mbstring, SPL, PDO, session, posix, readline, Reflection, standard, SimpleXML, pdo_sqlite, Phar, tokenizer, xml, xmlreader, xmlwriter, mysqlnd, cgi-fcgi, apcu, bcmath, exif, gd, gmp, imagick, intl, ldap, memcached, pcntl, pdo_mysql, pdo_pgsql, redis, sodium, zip, Zend OPcache

Nextcloud version: 20.0.7 - 20.0.7.1

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from:

Signing status

List of activated apps

``` Enabled: - accessibility: 1.6.0 - activity: 2.13.4 - admin_audit: 1.10.0 - analytics: 3.2.0 - apporder: 0.11.0 - bruteforcesettings: 2.0.1 - calendar: 2.1.3 - cloud_federation_api: 1.3.0 - comments: 1.10.0 - contacts: 3.4.3 - contactsinteraction: 1.1.0 - dashboard: 7.0.0 - dav: 1.16.2 - deck: 1.2.4 - documentserver_community: 0.1.8 - drawio: 0.9.8 - duplicatefinder: 0.0.4 - extract: 1.3.0 - federatedfilesharing: 1.10.2 - files: 1.15.0 - files_pdfviewer: 2.0.1 - files_readmemd: 1.2.0 - files_rightclick: 0.17.0 - files_sharing: 1.12.2 - files_trackdownloads: 1.9.0 - files_trashbin: 1.10.1 - files_versions: 1.13.0 - firstrunwizard: 2.9.0 - flowupload: 1.1.2 - groupfolders: 8.2.0 - guests: 1.6.2 - impersonate: 1.7.0 - issuetemplate: 0.7.0 - logreader: 2.5.0 - lookup_server_connector: 1.8.0 - metadata: 0.12.0 - nextcloud_announcements: 1.9.0 - notifications: 2.8.0 - oauth2: 1.8.0 - onlyoffice: 6.2.0 - password_policy: 1.10.1 - polls: 1.7.5 - privacy: 1.4.0 - provisioning_api: 1.10.0 - quickaccesssorting: 1.1.2 - quicknotes: 0.6.3 - quota_warning: 1.9.1 - recommendations: 0.8.0 - serverinfo: 1.10.0 - settings: 1.2.0 - sharebymail: 1.10.0 - sharepermissions: 1.0.1 - support: 1.3.0 - survey_client: 1.8.0 - suspicious_login: 3.2.1 - tasks: 0.13.6 - twofactor_admin: 3.0.0 - twofactor_backupcodes: 1.9.0 - twofactor_email: 2.0.0 - twofactor_nextcloud_notification: 3.1.0 - twofactor_totp: 5.0.0 - twofactor_u2f: 6.1.0 - twofactor_webauthn: 0.2.6 - viewer: 1.4.0 - whiteboard: 0.0.3 - workflowengine: 2.2.0 Disabled: - appointments - bbb - checksum - cms_pico - encryption - external - federation - files_3d - files_accesscontrol - files_antivirus - files_automatedtagging - files_downloadactivity - files_external - files_fulltextsearch - files_videoplayer - forms - fulltextsearch - mail - maps - ojsxc - phonetrack - photos - printer - richdocuments - smb_test - spreed - systemtags - text - theming - twofactor_gateway - updatenotification - user_ldap - user_status - weather_status - workflow_pdf_converter - workflow_script ```

Configuration (config/config.php)

``` { "memcache.local": "\\OC\\Memcache\\APCu", "apps_paths": [ { "path": "\/var\/www\/html\/apps", "url": "\/apps", "writable": false }, { "path": "\/var\/www\/html\/custom_apps", "url": "\/custom_apps", "writable": true } ], "memcache.locking": "\\OC\\Memcache\\Redis", "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 6379 }, "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "....ddns.net" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "overwrite.cli.url": "https:\/\/....ddns.net", "overwriteprotocol": "https", "dbtype": "mysql", "version": "20.0.7.1", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "theme": "", "loglevel": 0, "maintenance": false, "mail_smtpmode": "smtp", "mail_smtpauthtype": "LOGIN", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtpauth": 1, "mail_smtpsecure": "tls", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "app_install_overwrite": [ "calendar", "tasks", "spreed", "twofactor_email", "groupfolders" ] } ```

Are you using external storage, if yes which one:

Are you using encryption:

Are you using an external user-backend, if yes which one:

Client configuration

Browser:

Operating system:

Logs

Web server error log

``` ```

Nextcloud log

``` ```

Browser log

[CarlSchwan]

This should be solved by https://github.com/nextcloud/server/pull/29735 once this gets merged :)

[solracsf]

Should be fixed by https://github.com/nextcloud/server/pull/29735 Reopen a new issue if it is not.