Files are not deleted from S3 (primary)

solracsf commented 4 years ago

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
Subscribe to receive notifications on status change and new comments.

Steps to reproduce

Set S3 as primary storage
Upload, say, 2000 files into a folder (JPGs here)
Delete that folder, and try to empty trashbin

Expected behaviour

Trashbin should be empty correctly

Actual behaviour

After some time, an error appears (Error while empty trash). Reloading the page shows no more files either in the Files or Trashbin.

But files still on the Object Storage, here OBJECTS and SIZE:

Before these test operations (upload, delete...)

Following commands had been executed (after):

sudo -u testing php occ files:scan test

Starting scan for user 1 out of 1 (test)
+---------+-------+--------------+
| Folders | Files | Elapsed time |
+---------+-------+--------------+
| 0       | 0     | 00:00:00     |
+---------+-------+--------------+

sudo -u testing php occ files:cleanup

0 orphaned file cache entries deleted

sudo -u www-data php occ trashbin:cleanup --all-users

Remove deleted files for all users
Remove deleted files for users on backend Database
   test

One user has reported that interface show he is "using" 1,9Gb of storage, but he has NO FILES or FOLDERS at all, either in FILES or TRASHBIN in a production instance.

Server configuration

Operating system: Ubuntu 18.04

Web server: Nginx 17

Database: MariaDB 10.4

PHP version: 7.3

Nextcloud version: (see Nextcloud admin page) 18.0.3

Updated from an older Nextcloud/ownCloud or fresh install: Fresh install

Where did you install Nextcloud from: Official sources

Signing status:

Signing status

``` No errors have been found. ```

List of activated apps:

App list

``` Enabled: - accessibility: 1.4.0 - admin_audit: 1.8.0 - announcementcenter: 3.7.0 - apporder: 0.9.0 - cloud_federation_api: 1.1.0 - dav: 1.14.0 - external: 3.5.0 - federatedfilesharing: 1.8.0 - files: 1.13.1 - files_accesscontrol: 1.8.1 - files_automatedtagging: 1.8.2 - files_pdfviewer: 1.7.0 - files_rightclick: 0.15.2 - files_sharing: 1.10.1 - files_trashbin: 1.8.0 - files_versions: 1.11.0 - files_videoplayer: 1.7.0 - groupfolders: 6.0.3 - impersonate: 1.5.0 - logreader: 2.3.0 - lookup_server_connector: 1.6.0 - notifications: 2.6.0 - oauth2: 1.6.0 - password_policy: 1.8.0 - privacy: 1.2.0 - provisioning_api: 1.8.0 - settings: 1.0.0 - sharebymail: 1.8.0 - theming: 1.9.0 - theming_customcss: 1.5.0 - twofactor_backupcodes: 1.7.0 - viewer: 1.2.0 - workflow_script: 1.3.1 - workflowengine: 2.0.0 ```

Nextcloud configuration:

Config report

``` { "system": { "objectstore": { "class": "\\OC\\Files\\ObjectStore\\S3", "arguments": { "bucket": "testing.example.com", "autocreate": true, "key": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "hostname": "10.1.0.2", "port": 8080, "use_ssl": false, "region": "fr-par", "use_path_style": true } }, "log_type": "file", "logfile": "\/var\/log\/nextcloud\/testing.example.com-nextcloud.log", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "testing.example.com" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "18.0.3.0", "overwrite.cli.url": "https:\/\/testing.example.com", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "3306", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "dbdriveroptions": { "1009": "\/etc\/ssl\/mysql\/ca-cert.pem", "1008": "\/etc\/ssl\/mysql\/client-cert.pem", "1007": "\/etc\/ssl\/mysql\/client-key.pem", "1014": false }, "installed": true, "skeletondirectory": "", "default_language": "fr", "default_locale": "fr_FR", "activity_expire_days": 30, "auth.bruteforce.protection.enabled": false, "blacklisted_files": [ ".htaccess", "Thumbs.db", "thumbs.db" ], "htaccess.RewriteBase": "\/", "integrity.check.disabled": false, "knowledgebaseenabled": false, "logtimezone": "Europe\/Paris", "maintenance": false, "memcache.local": "\\OC\\Memcache\\APCu", "memcache.distributed": "\\OC\\Memcache\\Redis", "updatechecker": false, "appstoreenabled": false, "upgrade.disable-web": true, "filelocking.enabled": false, "overwriteprotocol": "https", "preview_max_scale_factor": 1, "redis": { "host": "***REMOVED SENSITIVE VALUE***", "port": 6379, "timeout": 2.5, "dbindex": 2, "password": "***REMOVED SENSITIVE VALUE***" }, "quota_include_external_storage": false, "theme": "", "trashbin_retention_obligation": "auto, 7", "updater.release.channel": "stable", "mail_smtpmode": "smtp", "mail_smtpsecure": "tls", "mail_sendmailmode": "smtp", "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtpauth": 1, "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "instanceid": "***REMOVED SENSITIVE VALUE***", "overwritehost": "testing.example.com", "preview_max_x": "1280", "preview_max_y": "800", "jpeg_quality": "70", "loglevel": 2, "enabledPreviewProviders": [ "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap" ], "apps_paths": [ { "path": "\/var\/www\/apps", "url": "\/apps", "writable": false }, { "path": "\/var\/www\/custom", "url": "\/custom_apps", "writable": true } ] } } ```

Logs are completely empty (we have just fired up a test instance, and test this use case).