Error handling folders via Mac OS Finder after E2EE successful enablement

ghost commented 3 years ago

How to use GitHub

Please use the 👍 reaction to show that you are affected by the same issue.
Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.

Steps to reproduce

I created a folder in the top level structure of my NC via iOS (folder Test) and I encrypted it with E2EE
I accessed the folder via desktop Finder (Mac OS) - all OK
I deleted the folder via desktop Finder - folder Test goes in the bin
the folder appears deleted in the Finder, however I get an error in the desktop sync app and in iOS the folder Test is still there in the top level
I moved back the folder Test from the bin into the Finder, desktop app syncs successfully
in iOS there is no way to delete the folder Test anymore: error 403
in iOS the folder is still accessible as well as its content

Expected behaviour

Removing the encrypted folder via the Finder in Mac OS should sync and remove the encrypted folder in iOS without errors.

Actual behaviour

I have a folder in my NC that I cannot move or delete or decrypt anymore

Server configuration

Operating system: Mac OS X Web server:

Database:

PHP version:

Nextcloud version: (see Nextcloud admin page) 19.0.4 Updated from an older Nextcloud/ownCloud or fresh install: updated from 19.0.3 Where did you install Nextcloud from:

Signing status:

Signing status

``` Login as admin user into your Nextcloud and access http://example.com/index.php/settings/integrity/failed paste the results here. ``` No errors have been found.

List of activated apps:

App list

``` If you have access to your command line run e.g.: sudo -u www-data php occ app:list from within your Nextcloud installation folder If not, provide at least the version number of the End-to-End-Encryption app. ``` Enabled: - accessibility: 1.5.0 - activity: 2.12.1 - admin_audit: 1.9.0 - bruteforcesettings: 2.0.1 - cloud_federation_api: 1.2.0 - comments: 1.9.0 - contactsinteraction: 1.0.0 - dav: 1.15.0 - encryption: 2.7.0 - end_to_end_encryption: 1.5.2 - federatedfilesharing: 1.9.0 - federation: 1.9.0 - files: 1.14.0 - files_antivirus: 3.0.0 - files_pdfviewer: 1.8.0 - files_rightclick: 0.16.0 - files_sharing: 1.11.0 - files_trashbin: 1.9.0 - files_versions: 1.12.0 - files_videoplayer: 1.8.0 - firstrunwizard: 2.8.0 - groupfolders: 7.1.1 - logreader: 2.4.0 - lookup_server_connector: 1.7.0 - nextcloud_announcements: 1.8.0 - notifications: 2.7.0 - oauth2: 1.7.0 - password_policy: 1.9.1 - photos: 1.1.0 - privacy: 1.3.0 - provisioning_api: 1.9.0 - quota_warning: 1.8.0 - ransomware_detection: 0.8.0 - ransomware_protection: 1.7.0 - recommendations: 0.7.0 - serverinfo: 1.9.0 - settings: 1.1.0 - sharebymail: 1.9.0 - support: 1.2.1 - suspicious_login: 3.2.1 - systemtags: 1.9.0 - text: 3.0.1 - theming: 1.10.0 - twofactor_admin: 3.0.0 - twofactor_backupcodes: 1.8.0 - twofactor_totp: 5.0.0 - unsplash: 1.1.7 - updatenotification: 1.9.0 - viewer: 1.3.0 - w2g2: 3.0.3 - workflowengine: 2.1.0 Disabled: - calendar - contacts - files_external - files_fulltextsearch - fulltextsearch - fulltextsearch_elasticsearch - mail - onlyoffice - spreed - survey_client - user_ldap

Nextcloud configuration:

Config report

``` If you have access to your command line run e.g.: sudo -u www-data php occ config:list system from within your Nextcloud installation folder or Insert your config.php content here. Make sure to remove all sensitive content such as passwords. (e.g. database password, passwordsalt, secret, smtp password, …) ``` { "system": { "instanceid": "***REMOVED SENSITIVE VALUE***", "passwordsalt": "***REMOVED SENSITIVE VALUE***", "secret": "***REMOVED SENSITIVE VALUE***", "trusted_domains": [ "REMOVED", "REMOVED" ], "datadirectory": "***REMOVED SENSITIVE VALUE***", "dbtype": "mysql", "version": "19.0.4.2", "overwrite.cli.url": "REMOVED", "dbname": "***REMOVED SENSITIVE VALUE***", "dbhost": "***REMOVED SENSITIVE VALUE***", "dbport": "", "dbtableprefix": "oc_", "mysql.utf8mb4": true, "dbuser": "***REMOVED SENSITIVE VALUE***", "dbpassword": "***REMOVED SENSITIVE VALUE***", "installed": true, "twofactor_enforced": "true", "twofactor_enforced_groups": [], "twofactor_enforced_excluded_groups": [], "enable_previews": true, "mail_smtpmode": "smtp", "mail_sendmailmode": "smtp", "mail_smtpauth": 1, "mail_from_address": "***REMOVED SENSITIVE VALUE***", "mail_domain": "***REMOVED SENSITIVE VALUE***", "mail_smtphost": "***REMOVED SENSITIVE VALUE***", "mail_smtpport": "587", "mail_smtpname": "***REMOVED SENSITIVE VALUE***", "mail_smtppassword": "***REMOVED SENSITIVE VALUE***", "mail_smtpauthtype": "LOGIN", "mail_smtpsecure": "tls", "memcache.local": "\\OC\\Memcache\\APCu", "maintenance": false, "theme": "", "loglevel": 2, "updater.release.channel": "stable", "activity_expire_days": 14, "auth.bruteforce.protection.enabled": true, "blacklisted_files": [ ".htaccess", "Thumbs.db", "thumbs.db" ], "cron_log": true, "enabledPreviewProviders": [ "OC\\Preview\\PNG", "OC\\Preview\\JPEG", "OC\\Preview\\GIF", "OC\\Preview\\BMP", "OC\\Preview\\XBitmap", "OC\\Preview\\Movie", "OC\\Preview\\PDF", "OC\\Preview\\MP3", "OC\\Preview\\TXT", "OC\\Preview\\MarkDown" ], "filesystem_check_changes": 0, "filelocking.enabled": "true", "htaccess.RewriteBase": "\/", "integrity.check.disabled": false, "knowledgebaseenabled": false, "logfile": "REMOVED", "logtimezone": "Europe\/Paris", "log_rotate_size": 104857600, "overwriteprotocol": "https", "preview_max_x": 1024, "preview_max_y": 768, "preview_max_scale_factor": 1, "quota_include_external_storage": false, "skeletondirectory": "" } }

Are you using external storage, if yes which one: local/smb/sftp/... No Are you using encryption: yes/no Yes Are you using an external user-backend, if yes which one: LDAP/ActiveDirectory/Webdav/... No

Client configuration

Browser: Brave/Chromium Operating system: Mac OS X 10.15.7

Logs

Web server error log

``` Insert your webserver log here ```

Nextcloud log (data/nextcloud.log)

Nextcloud log

``` Insert your Nextcloud log here ``` {"reqId":"46Jpzjev55zVGEviEuOe","level":3,"time":"2020-11-10T23:12:48+01:00","remoteAddr":"REMOVED","user":"alberto","app":"no app in context","method":"DELETE","url":"/ocs/v2.php/apps/end_to_end_encryption/api/v1/lock/509?format=json&e2e-token=tbqYej72dHn7Fmfp3uMQsFV3fHbqekzcYPn2Wr99cIWffOZSg34GW23AJRLEkzHW","message":{"Exception":"OCA\\EndToEndEncryption\\Exceptions\\MissingMetaDataException","Message":"Intermediate meta-data file missing","Code":0,"Trace":[{"file":"/var/www/nextcloud/apps/end_to_end_encryption/lib/Controller/LockingController.php","line":146,"function":"saveIntermediateFile","class":"OCA\\EndToEndEncryption\\MetaDataStorage","type":"->","args":["alberto",509]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":170,"function":"unlockFolder","class":"OCA\\EndToEndEncryption\\Controller\\LockingController","type":"->","args":[509]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":100,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\EndToEndEncryption\\Controller\\LockingController"},"unlockFolder"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":137,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->","args":[{"__class__":"OCA\\EndToEndEncryption\\Controller\\LockingController"},"unlockFolder"]},{"file":"/var/www/nextcloud/lib/private/AppFramework/Routing/RouteActionHandler.php","line":47,"function":"main","class":"OC\\AppFramework\\App","type":"::","args":["OCA\\EndToEndEncryption\\Controller\\LockingController","unlockFolder",{"__class__":"OC\\AppFramework\\DependencyInjection\\DIContainer"},{"id":"509","_route":"ocs.end_to_end_encryption.Locking.unlockFolder"}]},{"function":"__invoke","class":"OC\\AppFramework\\Routing\\RouteActionHandler","type":"->","args":[{"id":"509","_route":"ocs.end_to_end_encryption.Locking.unlockFolder"}]},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":297,"function":"call_user_func","args":[{"__class__":"OC\\AppFramework\\Routing\\RouteActionHandler"},{"id":"509","_route":"ocs.end_to_end_encryption.Locking.unlockFolder"}]},{"file":"/var/www/nextcloud/ocs/v1.php","line":88,"function":"match","class":"OC\\Route\\Router","type":"->","args":["/ocsapp/apps/end_to_end_encryption/api/v1/lock/509"]},{"file":"/var/www/nextcloud/ocs/v2.php","line":24,"args":["/var/www/nextcloud/ocs/v1.php"],"function":"require_once"}],"File":"/var/www/nextcloud/apps/end_to_end_encryption/lib/MetaDataStorage.php","Line":186,"CustomMessage":"--"},"userAgent":"Mozilla/5.0 (iOS) Nextcloud-iOS/3.0.12","version":"19.0.4.2"}

Browser log

``` Insert your browser log here, this could for example include: a) The javascript console log b) The network log c) ... ```

tobiasKaminsky commented 3 years ago

@marinofaggiana @er-vin

er-vin commented 3 years ago

Sounds like a duplicate, we got a report of the folder removal sometime failing on the desktop repository.

marinofaggiana commented 3 years ago

I can't make a test because the desktop app in my Macs don't works (don't synchronized correctly)

ghost commented 3 years ago

Just to provide an update, following various forums posts for solutions, I tried runnning occ files:scan --all; the command worked but it didn't solve the issue. There is discussion about manually unlocking files via mysql as temp fix, but I would like to avoid touching the db without you guys first looking into the issue - as this is linked to E2EE somehow, not just locked files

marinofaggiana commented 3 years ago

@albazilla you can look if exists yet a lock in your mysql and remove it without problem, can be dangerous only if another user change the metadata file

ghost commented 3 years ago

Easier solution to my locked encrypted folders:

log in the admin account web interface and disable the E2EE
log in the user web interface and manually delete the locked folders

Still, the E2EE is not stable as simply creating, moving, deleting files from desktop clients totally breaks the implementation. Secondly, there is also the issue about copying moving multiple files at once using the desktop clients, that breaks the desktop sync... I had to stop it and force it again multiple times to ensure all the files would be sync'd up. But this is now for another bug another time I feel like trusting E2EE again. Serious QA issues here guys.

joshtrichards commented 10 months ago

Many changes have taken place since this report, many surrounding matters like this - both here and at the client level. Since the OP is unreachable and this hasn't seen traffic in there years I'm closing is. Anyone experiencing similar scenario today, please create a dedicated Issue with your respect environment details. Thanks!

nextcloud / end_to_end_encryption