Invalid request on file upload on external SMB file share with encryption enabled

pstiegele commented 2 years ago

Steps to reproduce

Installed owncloud in docker container (using docker-compose) with this manual
Activating encryption (Encryption type: Master Key)
Adding an external storage (Type: SMB Personal (unique file IDs)) and enable encryption for this storage using the web ui.
Uploading a file which is bigger than 70KB to the external storage by using the owncloud windows client or the web upload

Expected behaviour

The upload should be successful without an error. The expected file should be uploaded to the external storage in an encrypted format.

Actual behaviour

When it's a small file of a few bytes (in my test a 2KB file was also succesful) the upload works fine. Is the file larger, than an error message appears in the owncloud client:

"Server hat "500 Internal Server Error" auf "PUT http://<owncloud_domain>/remote.php/dav/files/pstiegele/azstorage/IMG_20220509_100423.jpg" geantwortet (Invalid request for /IMG_20220509_100423.jpg.ocTransferId1513177531.part (InvalidArgumentException)),azstorage/IMG_20220509_100423.jpg

The upload also doesn't fail when encryption is disabled for the whole owncloud or for the external storage. So it seems to be a combination of enabled encryption and SMB file share.

Server configuration

Operating system: Ubuntu 20.04 in docker container

Web server: Apache

Database: MariaDB 10.6

PHP version: -

ownCloud version: 10.10.0.3

Updated from an older ownCloud or fresh install: fresh install

Where did you install ownCloud from: docker-compose, official owncloud server image

Signing status (ownCloud 9.0 and above):

No errors have been found.

The content of config/config.php: see gist

**List of activated apps:**

``` Enabled: - activity: - Version: 2.7.0 - Path: /var/www/owncloud/apps/activity - comments: - Version: 0.3.0 - Path: /var/www/owncloud/apps/comments - configreport: - Version: 0.2.1 - Path: /var/www/owncloud/apps/configreport - dav: - Version: 0.7.0 - Path: /var/www/owncloud/apps/dav - encryption: - Version: 1.5.1 - Path: /var/www/owncloud/apps/encryption - federatedfilesharing: - Version: 0.5.0 - Path: /var/www/owncloud/apps/federatedfilesharing - federation: - Version: 0.1.0 - Path: /var/www/owncloud/apps/federation - files: - Version: 1.5.2 - Path: /var/www/owncloud/apps/files - files_external: - Version: 0.9.0 - Path: /var/www/owncloud/apps/files_external - files_mediaviewer: - Version: 1.0.5 - Path: /var/www/owncloud/apps/files_mediaviewer - files_pdfviewer: - Version: 1.0.1 - Path: /var/www/owncloud/apps/files_pdfviewer - files_sharing: - Version: 0.14.0 - Path: /var/www/owncloud/apps/files_sharing - files_texteditor: - Version: 2.4.1 - Path: /var/www/owncloud/apps/files_texteditor - files_versions: - Version: 1.3.0 - Path: /var/www/owncloud/apps/files_versions - firstrunwizard: - Version: 1.2.0 - Path: /var/www/owncloud/apps/firstrunwizard - market: - Version: 0.6.3 - Path: /var/www/owncloud/apps/market - notifications: - Version: 0.5.4 - Path: /var/www/owncloud/apps/notifications - provisioning_api: - Version: 0.5.0 - Path: /var/www/owncloud/apps/provisioning_api - systemtags: - Version: 0.3.0 - Path: /var/www/owncloud/apps/systemtags - templateeditor: - Version: 0.4.0 - Path: /var/www/owncloud/apps/templateeditor - updatenotification: - Version: 0.2.1 - Path: /var/www/owncloud/apps/updatenotification Disabled: - admin_audit: - Path: /var/www/owncloud/apps/admin_audit - announcementcenter: - Path: /var/www/owncloud/apps/announcementcenter - customgroups: - Path: /var/www/owncloud/apps/customgroups - enterprise_key: - Path: /var/www/owncloud/apps/enterprise_key - external: - Path: /var/www/owncloud/apps/external - files_antivirus: - Path: /var/www/owncloud/apps/files_antivirus - files_classifier: - Path: /var/www/owncloud/apps/files_classifier - files_external_dropbox: - Path: /var/www/owncloud/apps/files_external_dropbox - files_external_ftp: - Path: /var/www/owncloud/apps/files_external_ftp - files_ldap_home: - Path: /var/www/owncloud/apps/files_ldap_home - files_lifecycle: - Path: /var/www/owncloud/apps/files_lifecycle - files_trashbin: - Path: /var/www/owncloud/apps/files_trashbin - firewall: - Path: /var/www/owncloud/apps/firewall - graphapi: - Path: /var/www/owncloud/apps/graphapi - guests: - Path: /var/www/owncloud/apps/guests - metrics: - Path: /var/www/owncloud/apps/metrics - oauth2: - Path: /var/www/owncloud/apps/oauth2 - openidconnect: - Path: /var/www/owncloud/apps/openidconnect - password_policy: - Path: /var/www/owncloud/apps/password_policy - ransomware_protection: - Path: /var/www/owncloud/apps/ransomware_protection - sharepoint: - Path: /var/www/owncloud/apps/sharepoint - systemtags_management: - Path: /var/www/owncloud/apps/systemtags_management - theme-enterprise: - Path: /var/www/owncloud/apps/theme-enterprise - user_external: - Path: /var/www/owncloud/apps/user_external - user_ldap: - Path: /var/www/owncloud/apps/user_ldap - user_shibboleth: - Path: /var/www/owncloud/apps/user_shibboleth - web: - Path: /var/www/owncloud/apps/web - windows_network_drive: - Path: /var/www/owncloud/apps/windows_network_drive - wopi: - Path: /var/www/owncloud/apps/wopi - workflow: - Path: /var/www/owncloud/apps/workflow ```

Are you using external storage, if yes which one: smb external storage (azure file share) (SMB Personal (unique file IDs))

Are you using encryption: yes, master key encryption

Are you using an external user-backend, if yes which one: no

Client configuration

Browser:

Operating system: Windows 10

Logs

Web server error log

ownCloud log (data/owncloud.log)

{"reqId":"bbd261c6-50ea-4acf-a953-633cf3e24c14",
"level":4,
"time":"2022-07-20T16:38:38+00:00",
"remoteAddr":"84.136.86.52",
"user":"pstiegele",
"app":"webdav",
"method":"MOVE",
"url":"\/remote.php\/dav\/uploads\/pstiegele\/848413445\/.file",
"message":"Caused by: {\"Exception\":\"Icewind\\\\SMB\\\\Exception\\\\InvalidArgumentException\",
\"Message\":\"Invalid request for \\\/a.b.ocTransferId1569500346.part (InvalidArgumentException)\",
\"Code\":22,
\"Trace\":
\"#0 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeState.php(75): Icewind\\\\SMB\\\\Exception\\\\Exception::fromMap()\\n#1 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeState.php(92): Icewind\\\\SMB\\\\Native\\\\NativeState->handleError()\\n#2 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeState.php(296): Icewind\\\\SMB\\\\Native\\\\NativeState->testResult()\\n#3 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeStream.php(145): Icewind\\\\SMB\\\\Native\\\\NativeState->write()\\n#4 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeWriteStream.php(59): Icewind\\\\SMB\\\\Native\\\\NativeStream->stream_write()\\n#5 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeWriteStream.php(46): Icewind\\\\SMB\\\\Native\\\\NativeWriteStream->flushWrite()\\n#6 [internal function]: Icewind\\\\SMB\\\\Native\\\\NativeWriteStream->stream_seek()\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(46): fseek()\\n#8 [internal function]: Icewind\\\\Streams\\\\Wrapper->stream_seek()\\n#9 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(46): fseek()\\n#10 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Stream\\\/Encryption.php(507): Icewind\\\\Streams\\\\Wrapper->stream_seek()\\n#11 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Stream\\\/Encryption.php(345): OC\\\\Files\\\\Stream\\\\Encryption->parentStreamSeek()\\n#12 [internal function]: OC\\\\Files\\\\Stream\\\\Encryption->stream_write()\\n#13 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(59): fwrite()\\n#14 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Stream\\\/Checksum.php(124): Icewind\\\\Streams\\\\Wrapper->stream_write()\\n#15 [internal function]: OC\\\\Files\\\\Stream\\\\Checksum->stream_write()\\n#16 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/helper.php(299): fwrite()\\n#17 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(208): OC_Helper::streamCopy()\\n#18 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/Directory.php(173): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put()\\n#19 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(307): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\Directory->createFile()\\n#20 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(133): Sabre\\\\DAV\\\\Tree->copyNode()\\n#21 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Tree.php(113): Sabre\\\\DAV\\\\Tree->copy()\\n#22 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Tree.php(163): OCA\\\\DAV\\\\Tree->copy()\\n#23 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Upload\\\/ChunkingPlugin.php(79): Sabre\\\\DAV\\\\Tree->move()\\n#24 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Upload\\\/ChunkingPlugin.php(58): OCA\\\\DAV\\\\Upload\\\\ChunkingPlugin->performMove()\\n#25 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(89): OCA\\\\DAV\\\\Upload\\\\ChunkingPlugin->beforeMove()\\n#26 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(603): Sabre\\\\DAV\\\\Server->emit()\\n#27 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(89): Sabre\\\\DAV\\\\CorePlugin->httpMove()\\n#28 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(472): Sabre\\\\DAV\\\\Server->emit()\\n#29 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(253): Sabre\\\\DAV\\\\Server->invokeMethod()\\n#30 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Server.php(345): Sabre\\\\DAV\\\\Server->start()\\n#31 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(31): OCA\\\\DAV\\\\Server->exec()\\n#32 \\\/var\\\/www\\\/owncloud\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')\\n#33 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Exception\\\/Exception.php\",\"Line\":44}"}

with part_file_in_storage = false it also fails:

{"reqId":"2db1e95f-5ae1-45e6-91cb-a6891d7046e3","level":4,"time":"2022-07-19T20:15:44+00:00","remoteAddr":"84.136.86.52","user":"pstiegele","app":"webdav","method":"PUT","url":"\/remote.php\/dav\/files\/pstiegele\/azure\/IMG_1130.jpg","message":"Caused by: {\"Exception\":\"Icewind\\\\SMB\\\\Exception\\\\InvalidArgumentException\",\"Message\":\"Invalid request for \\\/IMG_1130.jpg (InvalidArgumentException)\",\"Code\":22,\"Trace\":\"#0 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeState.php(75): Icewind\\\\SMB\\\\Exception\\\\Exception::fromMap()\\n#1 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeState.php(92): Icewind\\\\SMB\\\\Native\\\\NativeState->handleError()\\n#2 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeState.php(296): Icewind\\\\SMB\\\\Native\\\\NativeState->testResult()\\n#3 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeStream.php(145): Icewind\\\\SMB\\\\Native\\\\NativeState->write()\\n#4 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeWriteStream.php(59): Icewind\\\\SMB\\\\Native\\\\NativeStream->stream_write()\\n#5 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Native\\\/NativeWriteStream.php(46): Icewind\\\\SMB\\\\Native\\\\NativeWriteStream->flushWrite()\\n#6 [internal function]: Icewind\\\\SMB\\\\Native\\\\NativeWriteStream->stream_seek()\\n#7 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(46): fseek()\\n#8 [internal function]: Icewind\\\\Streams\\\\Wrapper->stream_seek()\\n#9 \\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/streams\\\/src\\\/Wrapper.php(46): fseek()\\n#10 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Stream\\\/Encryption.php(507): Icewind\\\\Streams\\\\Wrapper->stream_seek()\\n#11 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Stream\\\/Encryption.php(345): OC\\\\Files\\\\Stream\\\\Encryption->parentStreamSeek()\\n#12 [internal function]: OC\\\\Files\\\\Stream\\\\Encryption->stream_write()\\n#13 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/legacy\\\/helper.php(299): fwrite()\\n#14 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Encryption.php(812): OC_Helper::streamCopy()\\n#15 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Encryption.php(649): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Encryption->copyBetweenStorage()\\n#16 \\\/var\\\/www\\\/owncloud\\\/lib\\\/private\\\/Files\\\/Storage\\\/Wrapper\\\/Wrapper.php(579): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Encryption->moveFromStorage()\\n#17 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Connector\\\/Sabre\\\/File.php(271): OC\\\\Files\\\\Storage\\\\Wrapper\\\\Wrapper->moveFromStorage()\\n#18 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(1137): OCA\\\\DAV\\\\Connector\\\\Sabre\\\\File->put()\\n#19 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(492): Sabre\\\\DAV\\\\Server->updateFile()\\n#20 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/event\\\/lib\\\/WildcardEmitterTrait.php(89): Sabre\\\\DAV\\\\CorePlugin->httpPut()\\n#21 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(472): Sabre\\\\DAV\\\\Server->emit()\\n#22 \\\/var\\\/www\\\/owncloud\\\/lib\\\/composer\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(253): Sabre\\\\DAV\\\\Server->invokeMethod()\\n#23 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/lib\\\/Server.php(349): Sabre\\\\DAV\\\\Server->start()\\n#24 \\\/var\\\/www\\\/owncloud\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(31): OCA\\\\DAV\\\\Server->exec()\\n#25 \\\/var\\\/www\\\/owncloud\\\/remote.php(165): require_once('\\\/var\\\/www\\\/ownclo...')\\n#26 {main}\",\"File\":\"\\\/var\\\/www\\\/owncloud\\\/apps\\\/files_external\\\/3rdparty\\\/icewind\\\/smb\\\/src\\\/Exception\\\/Exception.php\",\"Line\":44}"}

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

pstiegele commented 1 year ago

It's still relevant.

github-actions[bot] commented 1 year ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 10 days if no further activity occurs. Thank you for your contributions.

github-actions[bot] commented 1 year ago

This issue has been automatically closed.

owncloud / core