[Bug]: Big files rush video limited to ~16Go max upload. (NextcloudPi - V28.0.2)

dwprod38 commented 8 months ago

⚠️ This issue respects the following points: ⚠️

[X] This is a bug, not a question or a configuration/webserver/proxy issue.
[X] This issue is not already reported on Github OR Nextcloud Community Forum (I've searched it).
[X] Nextcloud Server is up to date. See Maintenance and Release Schedule for supported versions.
[X] I agree to follow Nextcloud's Code of Conduct.

Bug description

Impossible to upload large files rush video more than 16Go despite setting "Upload Max Size" to 30Go in NextcloudPi Panel...

Steps to reproduce

Safari Browser:

Two attempts to upload Rush video large file (16,89Go) from "Upload Files" menu in dashboard (18 févr. 2024, 12:34:58 and 18 févr. 2024, 12:37:39)
Progression bar working fine (Displayed upload estimated time ~ 16min) with both attempts.
Three mins later, Nextcloud browser refresh itself with blank page and few seconds after, dashboard reappeared with fault message: "BadRequestExpected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 1564672 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side."

Nextcloud App iOS:

Upload big file (16,89Go) from "Upload Files" menu in iOS (18 févr. 2024, ~ 12:52:00)
Error message is displayed immediately as shown below:
No Log error message is registered in "nextcloud.log" linked with last upload attempt from iOS...

Expected behavior

With "Upload Max Size" set to 30Go, the 16,89Go rush file should be uploaded without problem with internal SSD 640Go free space on NextcloudPi with 64bits RaspberryPi 4 hardware (pictures below).

24-02-18 12-02-35 8907 24-02-18 12-02-21 8906

Installation method

Community NextcloudPi appliance

Nextcloud Server version

28

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.1

Web server

Apache (supported)

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

[X] Default user-backend (database)
[ ] LDAP/ Active Directory
[ ] SSO - SAML
[ ] Other

Configuration report

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "FR",
        "trusted_domains": {
            "0": "localhost",
            "7": "nextcloudpi",
            "5": "nextcloudpi.local",
            "8": "nextcloudpi.lan",
            "3": "nextcloudpi",
            "11": "***REMOVED SENSITIVE VALUE***",
            "1": "192.168.1.43",
            "2": "***REMOVED SENSITIVE VALUE***",
            "14": "nextcloudpi",
            "20": "***REMOVED SENSITIVE VALUE***"
        },
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "28.0.2.5",
        "overwrite.cli.url": "https:\/\/nextcloudpi\/",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "timeout": 0,
            "password": "***REMOVED SENSITIVE VALUE***"
        },
        "tempdirectory": "\/media\/myCloudDrive\/ncdatadwprod38\/data\/tmp",
        "mail_smtpmode": "sendmail",
        "mail_smtpauthtype": "LOGIN",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "preview_max_x": "2048",
        "preview_max_y": "2048",
        "jpeg_quality": "60",
        "overwriteprotocol": "https",
        "maintenance": false,
        "maintenance_window_start": 1,
        "logfile": "\/media\/myCloudDrive\/ncdatadwprod38\/data\/nextcloud.log",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "data-fingerprint": "***REMOVED SENSITIVE VALUE***",
        "loglevel": "2",
        "log_type": "file",
        "theme": "",
        "app_install_overwrite": [
            "camerarawpreviews",
            "workflow_media_converter"
        ]
    }
}

List of activated Apps

Enabled:
  - activity: 2.20.0
  - calendar: 4.6.5
  - camerarawpreviews: 0.8.4
  - circles: 28.0.0-dev
  - cloud_federation_api: 1.11.0
  - comments: 1.18.0
  - contacts: 5.5.1
  - contactsinteraction: 1.9.0
  - dashboard: 7.8.0
  - dav: 1.29.1
  - federatedfilesharing: 1.18.0
  - federation: 1.18.0
  - files: 2.0.0
  - files_external: 1.20.0
  - files_pdfviewer: 2.9.0
  - files_reminders: 1.1.0
  - files_sharing: 1.20.0
  - files_trashbin: 1.18.0
  - files_versions: 1.21.0
  - firstrunwizard: 2.17.0
  - logreader: 2.13.0
  - lookup_server_connector: 1.16.0
  - nextcloud_announcements: 1.17.0
  - notes: 4.9.2
  - notifications: 2.16.0
  - notify_push: 0.6.9
  - oauth2: 1.16.3
  - password_policy: 1.18.0
  - photos: 2.4.0
  - privacy: 1.12.0
  - provisioning_api: 1.18.0
  - recommendations: 2.0.0
  - related_resources: 1.3.0
  - serverinfo: 1.18.0
  - settings: 1.10.1
  - sharebymail: 1.18.0
  - support: 1.11.0
  - survey_client: 1.16.0
  - systemtags: 1.18.0
  - tasks: 0.15.0
  - text: 3.9.1
  - theming: 2.3.0
  - twofactor_backupcodes: 1.17.0
  - updatenotification: 1.18.0
  - user_status: 1.8.1
  - viewer: 2.2.0
  - weather_status: 1.8.0
  - workflow_media_converter: 1.9.3
  - workflowengine: 2.10.0
Disabled:
  - admin_audit: 1.18.0
  - bruteforcesettings: 2.8.0
  - encryption: 2.16.0
  - news: 24.0.0 (installed 24.0.0)
  - nextcloudpi: 0.0.2 (installed 0.0.1)
  - previewgenerator: 99.99.99 (installed 99.99.99)
  - suspicious_login: 6.0.0
  - twofactor_totp: 10.0.0-beta.2
  - user_ldap: 1.19.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

Error   no app in context   
BadRequest
Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 1564672 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

18 févr. 2024, 12:37:39 

Error   no app in context   
BadRequest
Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 9920512 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

18 févr. 2024, 12:37:39 

Error   no app in context   
BadRequest
Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 40960 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

18 févr. 2024, 12:34:58 

Error   no app in context   
BadRequest
Expected filesize of 10485760 bytes but read (from Nextcloud client) and wrote (to Nextcloud storage) 6348800 bytes. Could either be a network problem on the sending side or a problem writing to the storage on the server side.

18 févr. 2024, 12:34:58 

Error   no app in context   
Exception
HMAC does not match.
Could not decrypt or decode encrypted session data

18 févr. 2024, 12:00:51 

Error   no app in context   
Exception
HMAC does not match.
Could not decrypt or decode encrypted session data

18 févr. 2024, 12:00:51 

Error   no app in context   
Exception
HMAC does not match.
Could not decrypt or decode encrypted session data

18 févr. 2024, 09:14:48 

Error   no app in context   
Exception
HMAC does not match.
Could not decrypt or decode encrypted session data

18 févr. 2024, 09:14:48 

Error   no app in context   
Exception
HMAC does not match.
Could not decrypt or decode encrypted session data

18 févr. 2024, 09:14:48

Additional info

Large files like as ~12Go have been uploaded successfully...

dwprod38 commented 8 months ago

Hello everybody, here additional information:

After uploading NC from 28.0.2 to 28.0.3 and setting "nc-limit" upload size to 25Go instead of 30Go. I've attempted to upload 23.79Go rush video from iOS NC app (version 5.1.0.7). No Success as shown video below:

https://github.com/nextcloud/server/assets/95407859/f4aab764-a15c-46dc-afb2-fd869f02d415

Second upload attempt with iPadOS NC app: success !!!

Tonight I will attempt upload same file with browser

joshtrichards commented 3 months ago

The iOS error is a warning about storage space required on your iOS device to initiate the upload; it's not coming from Server and is wholly unrelated.

As for uploading via the web UI: it should be using chunking (unless you've disabled it). But there are numerous possibilities. It's generally a config/environment matter. However, you did originally report this early in the v28 release cycle and there were some issues in the uploader with some browser combinations. Those have since been fixed.

If you're still experiencing this in more recent versions, I would start by checking your browser inspector then looking at the Network tab and the Console tab to see what is going on during these upload transactions.

Further follow-up, unless likely to be a bug in Nextcloud itself, I would take to the help forum - https://help.nextcloud.com

nextcloud-command commented 2 months ago

This issue has been automatically marked as stale because it has not had recent activity and seems to be missing some essential information. It will be closed if no further activity occurs. Thank you for your contributions.

nextcloud / server