[Bug]: Large file upload to S3 storage fails

13hannes11 commented 1 year ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

Syncing large files from my nextcloud android app fails when trying to sync to S3 storage (Linode). I get the following error in the logs:

Declaration of Aws\HashingStream::seek($offset, $whence = Aws\SEEK_SET) must be compatible with Psr\Http\Message\StreamInterface::seek(int $offset, int $whence = Psr\Http\Message\SEEK_SET): void atnextcloud  /var/www/html/3rdparty/aws/aws-sdk-php/src/HashingStream.php#50

The full raw message:

{"reqId":"slTJq136n6Igj3Umee4T","level":3,"time":"2023-10-04T18:52:23+00:00","remoteAddr":"176.10.137.14","user":"13hannes11","app":"PHP","method":"MOVE","url":"/remote.php/dav/uploads/13hannes11/d4760723ce6fc3ac05aed57adb862eb5/.file","message":"Declaration of Aws\\HashingStream::seek($offset, $whence = Aws\\SEEK_SET) must be compatible with Psr\\Http\\Message\\StreamInterface::seek(int $offset, int $whence = Psr\\Http\\Message\\SEEK_SET): void at /var/www/html/3rdparty/aws/aws-sdk-php/src/HashingStream.php#50","userAgent":"Mozilla/5.0 (Android) Nextcloud-android/3.26.0","version":"27.1.1.0","data":{"app":"PHP"},"id":"651db4bca0c88"}

There is a similar issue that seems to have been closed #39981

Steps to reproduce

Upload a larger file from the web interface or android app to an external S3 storage folder from Linode.

Expected behavior

The file upload succeeds without issues.

Installation method

Community Docker image

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

None

Web server

None

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "overwriteprotocol": "https",
        "htaccess.RewriteBase": "\/",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "apps_paths": [
            {
                "path": "\/var\/www\/html\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/html\/custom_apps",
                "url": "\/custom_apps",
                "writable": true
            }
        ],
        "app_install_overwrite": [
            "ocr",
            "occweb",
            "keeweb",
            "files_clipboard",
            "files_downloadactivity",
            "socialsharing_email",
            "previewgenerator",
            "files_trackdownloads",
            "workflow_script",
            "files_external_onedrive",
            "impersonate",
            "richdocuments",
            "extract",
            "ransomware_protection",
            "webhooks"
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.kumeister.de",
            "nextcloud.hanneskuchelmeister.de"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "27.1.1.0",
        "overwrite.cli.url": "https:\/\/nextcloud.hanneskuchelmeister.de",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "data-fingerprint": "57570c17d6ffdb5400dac7eef52e22f7",
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_smtpport": "587",
        "mail_smtpauth": 1,
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpsecure": "tls",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
        "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 0,
        "updater.release.channel": "stable",
        "default_phone_region": "DE",
        "has_internet_connection": true,
        "theme": "",
        "overwritehost": "nextcloud.hanneskuchelmeister.de",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "enabledPreviewProviders": [
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\HEIC",
            "OC\\Preview\\Krita",
            "OC\\Preview\\XBitmap"
        ],
        "memories.exiftool": "\/var\/www\/html\/custom_apps\/memories\/exiftool-bin\/exiftool-amd64-glibc",
        "memories.vod.path": "\/var\/www\/html\/custom_apps\/memories\/exiftool-bin\/go-vod-amd64"
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - bruteforcesettings: 2.7.0
  - calendar: 4.5.2
  - camerarawpreviews: 0.8.2
  - checksum: 1.2.2
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contacts: 5.4.2
  - contactsinteraction: 1.8.0
  - cospend: 1.5.10
  - dashboard: 7.7.0
  - dav: 1.27.0
  - drawio: 2.1.2
  - extract: 1.3.6
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_accesscontrol: 1.17.1
  - files_automatedtagging: 1.17.0
  - files_downloadactivity: 1.16.0
  - files_external: 1.19.0
  - files_linkeditor: 1.1.16
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_retention: 1.16.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - fileslibreofficeedit: 1.1.0
  - flow_notifications: 1.7.0
  - gpoddersync: 3.8.1
  - gpxpod: 5.0.12
  - impersonate: 1.14.0
  - keeweb: 0.6.13
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - metadata: 0.19.0
  - music: 1.8.4
  - nextcloud_announcements: 1.16.0
  - nextpod: 0.7.3
  - notifications: 2.15.0
  - oauth2: 1.15.1
  - onlyoffice: 8.2.0
  - password_policy: 1.17.0
  - photos: 2.3.0
  - previewgenerator: 5.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - ransomware_protection: 1.14.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - side_menu: 3.10.3
  - socialsharing_email: 2.6.0
  - suspicious_login: 5.0.0
  - systemtags: 1.17.0
  - tasks: 0.15.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - twofactor_totp: 9.0.0
  - updatenotification: 1.17.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - webhooks: 0.4.3
  - workflow_script: 1.12.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - circles: 27.0.1 (installed 0.14.0)
  - encryption: 2.15.0
  - event_update_notification: 2.2.0 (installed 2.2.0)
  - files_scripts: 3.0.0 (installed 3.0.0)
  - files_trackdownloads: 1.11.0 (installed 1.11.0)
  - firstrunwizard: 2.16.0 (installed 2.4.0)
  - memories: 5.4.1 (installed 5.4.1)
  - storj: 0.0.9
  - support: 1.10.0 (installed 1.0.0)
  - survey_client: 1.15.0 (installed 1.3.0)
  - user_ldap: 1.17.0
  - user_status: 1.7.0 (installed 1.2.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

13hannes11 commented 1 year ago

The issue does not crop up when uploading files via the desktop application.

joshtrichards commented 1 year ago

https://github.com/nextcloud/server/issues/39981#issuecomment-1690144709

That other issue was closed out because the solution is documented in it and it's caused by an app you have installed. :-)

joshtrichards commented 1 year ago

FYI: The author of that relevant app is looking for testers for a patched release that addresses the issue: https://github.com/ariselseng/camerarawpreviews/issues/97

nextcloud / server