[Bug]: Server Side Encryption Does Not Encrypt Files When Using S3-Compatible Primary Storage

Algebro7 commented 2 years ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] Nextcloud Server is running on 64bit capable CPU, PHP and OS.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

I have a fresh install of the official nextcloud:fpm docker image using Digital Ocean Spaces as primary storage. As expected, the initial files are uploaded without encryption because I haven't toggled server-side encryption on yet, but after I installed the Default Encryption app and enabled server-side encryption, subsequent files are still showing up in plaintext in the bucket.

Steps to reproduce

Install nextcloud:fpm with docker
Configure the container to use Digital Ocean Spaces as primary storage with the following variables in docker-compose.yml:

          - OBJECTSTORE_S3_HOST=${NEXTCLOUD_S3_HOST}
          - OBJECTSTORE_S3_BUCKET=${NEXTCLOUD_S3_BUCKET}
          - OBJECTSTORE_S3_KEY=${NEXTCLOUD_S3_KEY}
          - OBJECTSTORE_S3_SECRET=${NEXTCLOUD_S3_SECRET}
          - OBJECTSTORE_S3_REGION=${NEXTCLOUD_S3_REGION}
          - OBJECTSTORE_S3_USE_PATH_STYLE=${NEXTCLOUD_S3_PATHSTYLE}
          - OBJECTSTORE_S3_SSL=${NEXTCLOUD_S3_SSL}
          - OBJECTSTORE_S3_PORT=${NEXTCLOUD_S3_PORT}

Finish setting up the app and install the Default Encryption module from the apps menu
Toggle Server-Side Encryption On
Upload Files
Log into Digital Ocean Spaces separately and observe that newly uploaded files are unencrypted:

spaces

Expected behavior

Files uploaded to the server after enabling server-side encryption are actually encrypted

Installation method

Official Docker image

Operating system

Debian/Ubuntu

PHP engine version

Other

Web server

Nginx

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Fresh Nextcloud Server install

Are you using the Nextcloud Server Encryption module?

Encryption is Enabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "password": "***REMOVED SENSITIVE VALUE***",
            "port": 6379
        },
        "objectstore": {
            "class": "\\OC\\Files\\ObjectStore\\S3",
            "arguments": {
                "bucket": "redacted",
                "key": "***REMOVED SENSITIVE VALUE***",
                "secret": "***REMOVED SENSITIVE VALUE***",
                "region": "nyc3",
                "hostname": "nyc3.digitaloceanspaces.com",
                "port": "443",
                "objectPrefix": "urn:oid:",
                "autocreate": false,
                "use_ssl": true,
                "use_path_style": false,
                "legacy_auth": false
            }
        },
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "cloud.redacted.com"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "24.0.3.2",
        "overwrite.cli.url": "https:\/\/cloud.redacted.com",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true
    }
}

List of activated Apps

Enabled:
  - accessibility: 1.10.0
  - activity: 2.16.0
  - bruteforcesettings: 2.4.0
  - circles: 24.0.0
  - cloud_federation_api: 1.7.0
  - comments: 1.14.0
  - contactsinteraction: 1.5.0
  - dashboard: 7.4.0
  - dav: 1.22.0
  - encryption: 2.12.0
  - federatedfilesharing: 1.14.0
  - federation: 1.14.0
  - files: 1.19.0
  - files_pdfviewer: 2.5.0
  - files_rightclick: 1.3.0
  - files_sharing: 1.16.2
  - files_trashbin: 1.14.0
  - files_versions: 1.17.0
  - files_videoplayer: 1.13.0
  - firstrunwizard: 2.13.0
  - logreader: 2.9.0
  - lookup_server_connector: 1.12.0
  - nextcloud_announcements: 1.13.0
  - notifications: 2.12.0
  - oauth2: 1.12.0
  - password_policy: 1.14.0
  - photos: 1.6.0
  - privacy: 1.8.0
  - provisioning_api: 1.14.0
  - recommendations: 1.3.0
  - serverinfo: 1.14.0
  - settings: 1.6.0
  - sharebymail: 1.14.0
  - support: 1.7.0
  - survey_client: 1.12.0
  - systemtags: 1.14.0
  - text: 3.5.1
  - theming: 1.15.0
  - twofactor_backupcodes: 1.13.0
  - updatenotification: 1.14.0
  - user_status: 1.4.0
  - viewer: 1.8.0
  - weather_status: 1.4.0
  - workflowengine: 2.6.0
Disabled:
  - admin_audit
  - files_external
  - user_ldap

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

Algebro7 commented 2 years ago

Adding a screenshot of the encryption settings:

FALErwi commented 2 years ago

The encryption isn't supported if you are using object storage as primary.

Algebro7 commented 2 years ago

The encryption isn't supported if you are using object storage as primary.

Ah interesting, can you point me to where that is in the documentation? I didn't see it anywhere

Algebro7 commented 2 years ago

Upon further testing, it looks like most files actually ARE encrypted, but I saw in the documentation that image thumbnails/previews etc are not encrypted, so that's possibly what I was seeing with images. One undocumented limitation I am seeing though is that when you create files in the web app editor (e.g., "New Text File"), those are silently uploaded to S3 without encryption.

szaimen commented 1 year ago

Hi, please update to 24.0.9 or better 25.0.3 and report back if it fixes the issue. Thank you!

My goal is to add a label like e.g. 25-feedback to this ticket of an up-to-date major Nextcloud version where the bug could be reproduced. However this is not going to work without your help. So thanks for all your effort!

If you don't manage to reproduce the issue in time and the issue gets closed but you can reproduce the issue afterwards, feel free to create a new bug report with up-to-date information by following this link: https://github.com/nextcloud/server/issues/new?assignees=&labels=bug%2C0.+Needs+triage&template=BUG_REPORT.yml&title=%5BBug%5D%3A+

nextcloud / server