[Bug]: Versions are broken on encrypted mounts of local or external storage

nils-one commented 1 year ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I use server-side encryption for mounts of some local mounted folders, but I also tested it with an FTP mount. If I sync files to these places, everything works well except versions. Try to access or to reset to an older version of the file end up in not found errors. For the non encrypted space, everything works well.

Steps to reproduce

Activate server-side encryption for external storages
As an Admin create a local or external mount and give at least one user access to it.
Put some files in this and edit them, update them, sync them. See that versions are created.
See at versions tab on the sidebar that only the current version is accessible. Older ones are broken.

Expected behavior

The versions should work like it does on the local storage.

Installation method

Community Web installer on a VPS or web space

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Enabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "27.0.2.1",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "skeletondirectory": "",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "dbindex": 1
        },
        "default_phone_region": "DE",
        "default_language": "de",
        "default_locale": "de_DE",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpmode": "smtp",
        "mail_sendmailmode": "smtp",
        "mail_smtpsecure": "ssl",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpauth": 1,
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "maintenance": false,
        "loglevel": 3,
        "theme": "",
        "encryption.legacy_format_support": false,
        "encryption.key_storage_migrated": false,
        "versions_retention_obligation": "auto, 32",
        "bulkupload.enabled": false,
        "quota_include_external_storage": true
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contactsinteraction: 1.8.0
  - dav: 1.27.0
  - encryption: 2.15.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - maps: 1.1.0
  - notifications: 2.15.0
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recognize: 4.3.2
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - updatenotification: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - bruteforcesettings: 2.7.0
  - circles: 27.0.1 (installed 27.0.1)
  - dashboard: 7.7.0 (installed 7.7.0)
  - nextcloud_announcements: 1.16.0 (installed 1.16.0)
  - suspicious_login: 5.0.0
  - twofactor_totp: 9.0.0
  - user_ldap: 1.17.0
  - weather_status: 1.7.0 (installed 1.7.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Empty regardless of the log level if I reproduce this error.

Additional info

The browser or operating system is not relevant. Tried it with Firefox on Linux and Windows 11 and also with Google Chrome on Windows. But I think it is a server-side problem.

joshtrichards commented 1 year ago

Hi @nils-one - Thanks for the report. Just a couple clarifications to make sure I understand the situation:

I use server-side encryption for mounts of some local mounted folders, but I also tested it with an FTP mount. If I sync files to these places, everything works well except versions. Try to access or to reset to an older version of the file end up in not found errors.

So both scenarios are External Storage, but one is using the Local backend and the other the FTP backend? i.e. https://docs.nextcloud.com/server/latest/admin_manual/configuration_files/external_storage_configuration_gui.html#available-storage-backends

Is that accurate?

For the non encrypted space, everything works well.

Non-encrypted meaning if you disable encryption on either of the External Mounts or do you mean files simply stored in your configured datadirectory without any mounting involved?

Thanks again for your report.

nils-one commented 1 year ago

Sorry for the confusing description. I try to clarify: Both scenarios where version are not working are external mounts. One with local backend and one with FTP backend. With encryption on. If I store the file on the datadirectory without mounting involved and without encryption, it works. If I switch on encryption for datadirectory too, versions also work. What I've not tested yet is using mounting without encryption, as I don't want to switch encryption completely off (as a lot of files would be affected). But maybe I can set up a test instance later.

Simthem commented 1 year ago

Hi, If I can give some of my personnal experience with this issue :

With external storage directly connected by request (ftp or ssh) -> I have versionning issues when encryption is enabled on it
With external storage directly connected by request (ftp or ssh) -> no issue without encryption disabled on it
With external storage mounted (with sshfs for example) -> I have versionning issues when encryption is enabled on it
With external storage mounted (with sshfs for example) -> no issue without encryption disabled on it

Nextcloud version : 27.1.1

Thanks a lot for all your support :)

nextcloud / server