Downloading, sharing, modifying older (encrypted) images fails, displaying works

cpm1 commented 1 year ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

server side encryption is enabled.

I have some images stored since 2019. Displaying them within Nextcloud works as expected, but downloading them results in a file which is correctly named, but contains the description of a 500 server error. The Nextcloud desktop client reports additionally

Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.

during sync operation.

Note that this does not happen with newly added images, just with older ones. Also, it might be related to https://github.com/nextcloud/richdocuments/issues/2996 .. the symptoms sound similar.

With this bug, it is not possible to download any older images, which is quite a major issue. I can't say when exactly it started, as I didn't try to access those images in a while, but I seem to remember it working in v26.

Steps to reproduce

Got to photos.
Select to download an older photo.
Observe webserver error message being downloaded instead of real image and various error messages.

Expected behavior

Selected photo is downloaded correctly.

Installation method

Community Manual installation with Archive

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Apache (supported)

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Enabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "installed": true,
        "dbtype": "mysql",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "forcessl": true,
        "blacklisted_files": [
            ".htaccess"
        ],
        "overwritehost": "",
        "overwriteprotocol": "",
        "overwritewebroot": "",
        "overwritecondaddr": "",
        "proxy": "",
        "proxyuserpwd": "***REMOVED SENSITIVE VALUE***",
        "theme": "",
        "default_language": "de",
        "default_phone_region": "de",
        "3rdpartyroot": "",
        "3rdpartyurl": "",
        "defaultapp": "files",
        "knowledgebaseenabled": true,
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpdebug": false,
        "mail_smtpmode": "sendmail",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": 25,
        "mail_smtptimeout": 10,
        "mail_smtpsecure": "",
        "mail_smtpauth": false,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "filelocking.enabled": true,
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "trashbin_retention_obligation": "30, auto",
        "allow_user_to_change_display_name": true,
        "appcodechecker": "",
        "updatechecker": true,
        "check_for_working_htaccess": true,
        "log_type": "file",
        "loglevel": 2,
        "logdateformat": "F d, Y H:i:s",
        "logtimezone": "Europe\/Berlin",
        "remember_login_cookie_lifetime": 1296000,
        "session_lifetime": 86400,
        "custom_csp_policy": "default-src 'self'; script-src 'self' 'unsafe-eval'; style-src 'self' 'unsafe-inline'; frame-src *; img-src *; font-src 'self' data:; media-src *",
        "xframe_restriction": true,
        "maintenance": false,
        "apps_paths": [
            {
                "path": "\/var\/www\/nextcloud\/apps",
                "url": "\/apps",
                "writable": true
            }
        ],
        "customclient_desktop": "",
        "customclient_android": "",
        "customclient_ios": "",
        "enable_previews": true,
        "preview_max_scale_factor": 10,
        "preview_libreoffice_path": "\/usr\/bin\/libreoffice",
        "preview_office_cl_parameters": "",
        "enable_avatars": true,
        "openssl": [],
        "singleuser": false,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "version": "27.0.2.1",
        "trusted_domains": [
            "***REMOVED SENSITIVE VALUE***"
        ],
        "secret": "***REMOVED SENSITIVE VALUE***",
        "overwrite.cli.url": "***REMOVED SENSITIVE VALUE***",
        "updater.release.channel": "stable",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "encryption.key_storage_migrated": false
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - calendar: 4.4.4
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contacts: 5.3.2
  - dashboard: 7.7.0
  - dav: 1.27.0
  - encryption: 2.15.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_external: 1.19.0
  - files_pdfviewer: 2.8.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - nextcloud_announcements: 1.16.0
  - notifications: 2.15.0
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_email: 2.7.3
  - twofactor_nextcloud_notification: 3.7.0
  - updatenotification: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - bruteforcesettings: 2.7.0 (installed 1.3.0)
  - contactsinteraction: 1.8.0 (installed 1.5.0)
  - suspicious_login: 5.0.0
  - twofactor_totp: 9.0.0
  - user_ldap: 1.17.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{
    "app": "no app in context",
    "data": [
    ],
    "level": 3,
    "message": "Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.",
    "method": "GET",
    "remoteAddr": REMOTE-IP,
    "reqId": "ZwUzYvaInJ9Lkoj2bj1P",
    "time": "August 25, 2023 12:26:09",
    "url": "/nextcloud/remote.php/dav/files/USERNAME/Photos/IMG_5568.JPG",
    "user": USERNAME,
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15",
    "version": "27.0.2.1"
}
{
    "app": "webdav",
    "exception": {
        "Code": 0,
        "CustomMessage": "Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.",
        "Exception": "OC\Encryption\Exceptions\DecryptionFailedException",
        "exception": {
        },
        "File": "/var/www/nextcloud/apps/encryption/lib/Crypto/Encryption.php",
        "Hint": "Diese Datei kann nicht entschlüsselt werden, es handelt sich wahrscheinlich um eine geteilte Datei. Bitte kontaktieren Sie den Eigentümer der Datei und bitten Sie darum, die Datei noch einmal mit Ihnen zu teilen.",
        "Line": 398,
        "message": "Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.",
        "Message": "Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.",
        "Trace": [
            {
                "args": [
                    "*** sensitive parameters replaced ***"
                ],
                "class": "OCA\Encryption\Crypto\Encryption",
                "file": "/var/www/nextcloud/lib/private/Files/Stream/Encryption.php",
                "function": "decrypt",
                "line": 517,
                "type": "->"
            },
            {
                "class": "OC\Files\Stream\Encryption",
                "file": "/var/www/nextcloud/lib/private/Files/Stream/Encryption.php",
                "function": "readCache",
                "line": 316,
                "type": "->"
            },
            {
                "class": "OC\Files\Stream\Encryption",
                "function": "stream_read",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/Wrapper.php",
                "function": "fread",
                "line": 55
            },
            {
                "class": "Icewind\Streams\Wrapper",
                "file": "/var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php",
                "function": "stream_read",
                "line": 96,
                "type": "->"
            },
            {
                "class": "Icewind\Streams\CallbackWrapper",
                "function": "stream_read",
                "type": "->"
            },
            {
                "file": "/var/www/nextcloud/3rdparty/sabre/http/lib/Sapi.php",
                "function": "stream_copy_to_stream",
                "line": 110
            },
            {
                "class": "Sabre\HTTP\Sapi",
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "function": "sendResponse",
                "line": 490,
                "type": "::"
            },
            {
                "class": "Sabre\DAV\Server",
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "function": "invokeMethod",
                "line": 253,
                "type": "->"
            },
            {
                "class": "Sabre\DAV\Server",
                "file": "/var/www/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
                "function": "start",
                "line": 321,
                "type": "->"
            },
            {
                "class": "Sabre\DAV\Server",
                "file": "/var/www/nextcloud/apps/dav/lib/Server.php",
                "function": "exec",
                "line": 364,
                "type": "->"
            },
            {
                "class": "OCA\DAV\Server",
                "file": "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php",
                "function": "exec",
                "line": 35,
                "type": "->"
            },
            {
                "args": [
                    "/var/www/nextcloud/apps/dav/appinfo/v2/remote.php"
                ],
                "file": "/var/www/nextcloud/remote.php",
                "function": "require_once",
                "line": 172
            }
        ]
    },
    "level": 3,
    "message": "Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.",
    "method": "GET",
    "remoteAddr": REMOTE-IP,
    "reqId": "ZwUzYvaInJ9Lkoj2bj1P",
    "time": "August 25, 2023 12:26:09",
    "url": "/nextcloud/remote.php/dav/files/USERNAME/Photos/IMG_5568.JPG",
    "user": USERNAME,
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.6 Safari/605.1.15",
    "version": "27.0.2.1"
}

Additional info

nextcloud.log output repeats for every tried/failed image. IPs and usernames removed.

juliushaertl commented 1 year ago

I don't see a richdocuments relation, transferring back

juliushaertl commented 1 year ago

Might still be fixed already with https://github.com/nextcloud/server/pull/39447 but would only affect new files or reuploaded ones.

joshtrichards commented 1 year ago

Hi @cpm1 - The error message you're receiving:

Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.

...comes up when the File key file for the associated file isn't found.

Are you using a master key or per-user keys?
Was the parent folder (or any of the images themselves) originally shared with the account experiencing this problem?
Can you run occ encryption:scan:legacy-format?

cpm1 commented 1 year ago

Sorry for not replying earlier ..

Are you using a master key or per-user keys?

master key

Was the parent folder (or any of the images themselves) originally shared with the account experiencing this problem?

Exactly. I created that folder back in 2019, shared it via link with several people and recently realised that nobody can download the images anymore. Yet they show just fine in the in-line image viewer of nextcloud.

Can you run [occ encryption:scan:legacy-format]

Sure, here's the output:

# sudo -u www-data php ./occ encryption:scan:legacy-format
Scanning all files for legacy encryption
Scanning all files for cpm1
All scanned files are properly encrypted. You can disable the legacy compatibility mode.

floss4good commented 7 months ago

In case you need more details for reproducing this issue I am going to describe a scenario that I encountered recently.

Few months ago (at that time the Nextcloud server version was probably 25.0.5):

I have uploaded a folder - let's call it folder_A - with the following simplified structure:

folder_A
├─folder_AA
│ ├─folder_AAA
│ │ └─<old_files_1>
│ └─folder_AAB
│   └─<old_files_2>
└─<old_files_3>

If it matters, from what I remember I've used the Desktop app for this.
From web I created a password protected Share link (just 'Read' permission).

About a week ago (meanwhile the server was updated to 27.1.5):

I have created a new child folder - folder_AB - inside which I moved all the files stored directly within parent folder_A.
I've also created another child folder - folder_AC - and uploaded some new files within it.

The folders structure is now the below one:

folder_A
├─folder_AA
│ ├─folder_AAA
│ │ └─<old_files_1>
│ └─folder_AAB
│   └─<old_files_2+new_file>
├─folder_AB
│ └─<old_files_3>
└─folder_AC
└─<new_files>

Further, I have renamed a file from folder_AAB and also uploaded a new file here.
If it matters, this time I've made all the changes from the web client.

Now:

The Desktop client (v3.11.0) started to show the following error for all the files from folder_AB (the ones moved from root folder to a newly created folder):
- Connection closed (Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.)
The other files seems to be synchronized.
From web, logged in as the owner of the files:
- I can view and preview (probably) all the files/photos;
- I can download files that were uploaded recently;
- When trying to download files that were uploaded few months ago a new empty browser tab is opened and nothing happens. In the browser's console a 500 error is logged while in 'Network' tab the request has Blocked status and 'Transferred' column says 'NS_ERROR_WONT_HANDLE_CONTENT'.
Using the share link:
- When entering a folder I can see in the console a 404 error for domain.tld/ocs/v2.php/apps/text/public/workspace?path=...&shareToken=...;
- I can view and preview (probably) all the files/photos;
- I can download files that were uploaded recently;
- When trying to download old files, it depends on the browser; if 'Always ask you where to save files' Firefox setting is not enabled (and the file should be automatically saved):
- The browser shows that the download failed;
- An empty file is created with the expected name;
- A temp .part file is also created but it is actually a HTML with the following error:
  - Error Cannot download file Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.
- If 'Always ask you where to save files' Firefox setting is enabled:
- An empty file is created with the expected name;
- A browser Browser alert with a message like this:
  - /path/.jpeg.part could not be saved, because the source file could not be read. Try again later, or contact the server administrator.
- For images the preview can be downloaded with right click → 'Save Image As...'.
- Selecting multiple files (or a whole folder) and using Download action will download the zip but it is actually the same HTML with the above mentioned 'Cannot download file' error.

Tried some things (to see what happens) but nothing changed concerning the download:

Renamed one of the files from folder_AAB.
Moved one of the files from folder_AAB back into folder_A.
Created a second Share link (password protected, View only).

joshtrichards commented 3 months ago

Possibly relevant: #45669

floss4good commented 2 months ago

Indeed, seems like the issue is fixed in v27.1.11. Thank you @come-nc

floss4good commented 2 months ago

For the record, while testing the fix for #46196 I've noticed that the Cannot decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you. error is being logged together with the issue while running UpdateSingleMetadata warning for 'no app in context' on a fresh 29.0.3 server instance, without any shares created.

nextcloud / server