[Bug]: Restoring multiple big folders from trash bin fails

mgallien commented 5 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

Restoring multiple big folders at once from trash bin fails with an error message: "Failed to restore some elements"

Steps to reproduce

create 4 folders named bigFolder1, bigFolder2, bigFolder3 and bigFolder4
create many empty files inside each folder for i in {1..10000}; do touch test${i}.txt; done
get the files scan to finish sudo -u www-data php ./occ files:scan <uid>
delete the 4 folders from web UI
try to restore them with trash bin web UI

Expected behavior

all folders are restored without error

Installation method

Community Manual installation with Archive

Nextcloud Server version

master

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[ ] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

sudo -u www-data php occ config:list system 
{
    "system": {
        "debug": true,
        "profiler": true,
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/apps-extra",
                "url": "\/apps-extra",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/apps-shared",
                "url": "\/apps-shared",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/apps-writable",
                "url": "\/apps-writable",
                "writable": true
            }
        ],
        "allow_local_remote_servers": true,
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "1025",
        "skeletondirectory": "\/skeleton",
        "setup_create_db_user": false,
        "loglevel": 0,
        "log_query": false,
        "query_log_file": "\/var\/www\/html\/data\/query.log",
        "query_log_file_requestid": "yes",
        "diagnostics.logging": true,
        "diagnostics.logging.threshold": 0,
        "log.condition": {
            "apps": [
                "diagnostics",
                "admin_audit"
            ]
        },
        "bulkupload.enabled": false,
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "nextcloud",
            "nextcloud.local",
            "192.168.21.6",
            "localhost"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.0.5",
        "overwrite.cli.url": "http:\/\/nextcloud.local",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "git",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "lookup_server": "",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***"
    }
}

List of activated Apps

sudo -u www-data php occ app:list
Enabled:
  - cloud_federation_api: 1.12.0
  - comments: 1.19.0
  - contactsinteraction: 1.10.0
  - dashboard: 7.9.0
  - dav: 1.30.0
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - lookup_server_connector: 1.17.0
  - oauth2: 1.17.0
  - provisioning_api: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - systemtags: 1.19.0
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - updatenotification: 1.19.0
  - user_oidc: 1.3.6
  - user_status: 1.9.0
  - weather_status: 1.9.0
  - workflowengine: 2.11.0
Disabled:
  - admin_audit: 1.19.0
  - encryption: 2.17.0
  - files_external: 1.21.0
  - testing: 1.19.0
  - user_ldap: 1.20.0

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

mgallien commented 5 months ago

@sorbaugh

sorbaugh commented 5 months ago

This reminds me a bit of a reported issue where an instance runs out of memory when doing operations on super large directories. This is very infrastructure/setup dependant, and It might be unrelated, but an idea then was to create some sort of admin setting to limit the number of allowed files in a directory, including the trashbin. While certainly not a satisfactory solution, we're bound to run into some kind of friction when dealing with absurdly large amounts of files/data.

Depending on what analysis of the root cause reveals, maybe this could be an alternative approach.

sorbaugh commented 5 months ago

Looping in @come-nc & @Altahrim