nextcloud / desktop

💻 Desktop sync client for Nextcloud
https://nextcloud.com/install/#install-clients
GNU General Public License v2.0
3.01k stars 791 forks source link

"Connection closed" message when syncing files larger then +- 100Mb #4278

Open Jhutjens92 opened 2 years ago

Jhutjens92 commented 2 years ago

I have my nextcloud installation running in a docker container. It's connected to a MySQL DB (another docker container) and exposed to the web using SWAG. All the relevant php.ini/config files have the following settings

Uploading via webserver is no problem but whenever i try to sync the same file using the windows sync client i receive a "Connection Closed"

Expected behaviour

Files should just upload to the nextcloud server.

Actual behaviour

Files aren't being uploaded and client throws an error:

Steps to reproduce

  1. Run nextcloud in docker container.
  2. Use SWAG (docker container) to link (sub)domain to nextcloud
  3. Upload large file (larger then 100Mb) (in my case a .mp4)
  4. See response in client log.

Client configuration

Client version: 3.4.2

Operating system: Microsoft Windows 10 Pro (10.0.19041 Build 19041)

OS language: Dutch

Installation path of client: C:\Program Files\Nextcloud

Nextcloud version: Nextcloud Hub II (23.0.0)

Storage backend: Local server storage

Logs

  1. Client logfile: Client_20220213_2039_owncloud.log.0.txt

  2. Web server error log: N.A.

  3. Server logfile: nextcloud log (data/nextcloud.log): nextcloud.log

(ignore the failed login)

mgallien commented 2 years ago

so if I understand it correctly there is o way to tune the configuration without editing the text configuration file ?

rforced commented 2 years ago

I agree, this should be a setting exposed in the GUI to make it easier for people to change it

citizenserious commented 2 years ago

@Ventilgummi You are genius! Thanks so much. Works like a champ.

ghost commented 2 years ago

This worked but needs to be built into the application or enabled at server level because I have other users on the server and they would have to do this on their PC's

github-actions[bot] commented 2 years ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

rforced commented 2 years ago

Can this be updated to a feature request?

Samael774 commented 2 years ago

Hi everybody, thanks for finding this. I found simply setting the maximum chunk size to 50 MB (half of Cloudflare's 100 MB upload size limit) worked to resolve this issue. I put together a short guide to fix this issue with the latest stable release (3.4.4, but should work on any client v3.4+). I tried to make it as accessible as possible to follow.

Windows Fix

Press Win+R on your keyboard to open the Run application. Past the following in the dialog box: %APPDATA%\Nextcloud\nextcloud.cfg This will either ask you to pick an application to open nextcloud.cfg or will open in your default text editor (unless you have something else set to open .cfg files). If it asks you to pick an application, feel free to use Notepad or any other editor. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

MacOS Fix

Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box: $HOME/Library/Preferences/Nextcloud Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

Linux Fix

Open a terminal window and edit the following file: nano $HOME/.config/Nextcloud/nextcloud.cfg Add the following line under the [General] section: maxChunkSize=50000000 Save the file (Ctl+o, Ctl+x), then quit Nextcloud desktop, and start it again.

Thanks! This worked. Can confirm that it's still an issue on Windows app. I'm running NC as a dockercontainer, Traefik as reverse proxy and Cloudflare DNS with proxy

For me works too (Arch Linux). Thanks.

EagleonePrimo commented 2 years ago

Hi everybody, thanks for finding this. I found simply setting the maximum chunk size to 50 MB (half of Cloudflare's 100 MB upload size limit) worked to resolve this issue. I put together a short guide to fix this issue with the latest stable release (3.4.4, but should work on any client v3.4+). I tried to make it as accessible as possible to follow.

Windows Fix

Press Win+R on your keyboard to open the Run application. Past the following in the dialog box: %APPDATA%\Nextcloud\nextcloud.cfg This will either ask you to pick an application to open nextcloud.cfg or will open in your default text editor (unless you have something else set to open .cfg files). If it asks you to pick an application, feel free to use Notepad or any other editor. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

MacOS Fix

Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box: $HOME/Library/Preferences/Nextcloud Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

Linux Fix

Open a terminal window and edit the following file: nano $HOME/.config/Nextcloud/nextcloud.cfg Add the following line under the [General] section: maxChunkSize=50000000 Save the file (Ctl+o, Ctl+x), then quit Nextcloud desktop, and start it again.

Thanks! This worked. Can confirm that it's still an issue on Windows app. I'm running NC as a dockercontainer, Traefik as reverse proxy and Cloudflare DNS with proxy

It work for me too. i'm use Nextcloud AIO / Nextcloud Desktop Windows

josfaber commented 2 years ago

Desktop client 3.6.0 on mac, same issue. Previous version also had the issue. Big files >500mb just don't come through. Solutions in nextcloud.cfg mentioned above don't solve the issue.

DoctorMcKay commented 2 years ago

Also having this issue with 3.6.0 on Windows. There really needs to be a way to control the client's chunking behavior from the server. Telling people "just don't use Cloudflare, or pay thousands of dollars for an enterprise account to raise your upload limit to 500 MB" isn't really a solution.

josfaber commented 2 years ago

I ended up reinstalling the client and updating (minor version) and rebooting the server. Then let it do it's thing for an hour and checked it, no more problems for now. It's installed on my default Debian thinclient, I did not upgrade the upload limit. I would think (and hope) nextcloud uses chuncked uploading?

originalfoo commented 2 years ago

Getting same error. Editing config file as stated in thread above (and restarting Windows), reinstalling client, etc., doesn't fix it.

When downloading files > ~60 MB from nc server it gives "Connection Closed" error every time.

If the error at least gave more info - eg. how many MB was the connection closed at, or how much time had it spent before connection closed, I might have some info to help track down source of the problem.

github-actions[bot] commented 1 year ago

This bug report did not receive an update in the last 4 weeks. Please take a look again and update the issue with new details, otherwise the issue will be automatically closed in 2 weeks. Thank you!

ThatTallGuy21 commented 1 year ago

Bump. Looks like this still isn't fixed, and the work around may not be working anymore?

rforced commented 1 year ago

I think a nice addition that would resolve this issue is expose the option to change the chunk size from the GUI.

ThatTallGuy21 commented 1 year ago

Can anyone that works on Nextcloud speak to how and if this can be an included feature? If that's all that's needed at this point then can this be closed and moved over to a feature request thread? Really surprised that this is just sitting out there untouched by the dev team. It seems like a really common use case that should be a lot easier to handle.

Valdnet commented 1 year ago

ping @nextcloud/desktop

Jhutjens92 commented 1 year ago

It seems like this issue isn't being picked up? 9 months and counting.

mstrandbo commented 1 year ago

Needs to be a server side setting, since we can't expect all users to know or understand how to set this setting in the configfile. They wouldn't even necessarily know the server is behind Cloudflare.

ThatTallGuy21 commented 1 year ago

It's super disappointing to see how unresponsive this topic has gone. It seems like an incredibly reasonable feature, yet the radio silence from those that support NC is shocking. If something this straightforward can't be addressed or even updates provided, then I'm going to need to look at other options elsewhere. This seems incredibly unreasonable.

MP715 commented 1 year ago

Having the same problem. Running client version 3.6.2 on Windows 11. Most of the time I get "Connection Closed" but sometimes I get "413 Payload Too Large". I feel like I've tried everything. UPDATE: The settings below seem to work for me. Albeit very slowly. 2 hours for 10 gigs. Tried tweaking many ways and if I add another digit, it dies. Really stinks when I'm in my own home and I can't use LAN speed. How can I connect locally?

chunkSize=999999 minChunkSize=999999 maxChunkSize=999999

marcosmolla commented 1 year ago

Removing one zero worked for me: maxChunkSize=5000000

But I agree, this is nothing that the user should have to worry about. And IF at all then from the GUI.

vithusel commented 1 year ago

does the maxChunkSize need to be reset after every update to the Desktop client? Or does it persist through updates?

Lightning2X commented 1 year ago

It persists through updates as I've updated quite a lot of times since then and haven't had the issue since

matissime commented 1 year ago

Worked for me! This should be included in the settings of the Nextcloud Dekstop app with automatic detection if necessary.

MP715 commented 1 year ago

What I want to know is, is this a problem with the desktop app? The nextcloud server? Cloudflare limitations?

jospoortvliet commented 1 year ago

@MP715 it's in cloudflare, but the client could/should ideally work around it.

https://github.com/nextcloud/desktop/pull/4826 is a solution - testing and posting results of that there would be helpful - so those who care about moving this forward, consider testing and providing feedback there.

Also, thanks to @jkellerer for their work on it 👍🏻

vithusel commented 1 year ago

Somewhat related to this topic. Ill be making a script in the next few weeks to automate adding maxChunkSize to the windows client config as theres a few users who will get affected by this when i get my servers to run via cloudflare tunnel.

Has anyone had any issues with NextCloud Talk or uploading files via the web portal when using cloudflare? Thought i'd ask before spending hours testing everything myself.

whitephoenix117 commented 1 year ago

Figured I would add my experience

Problem "413" Error after uploading ~1GB into a 22GB file using desktop Version 3.6.4 (macOS) Error was NOT reproducible when uploading via web.

I have "experimental" virtual file support enabled. I use cloudflare for my DNS but do NOT use proxy. I use Let's Encrypt CA for SSL.

Upload was on LAN

Not Working Config

[General] clientVersion=3.6.4git (build 12826) confirmExternalStorage=true newBigFolderSizeLimit=500 optionalServerNotifications=true showCallNotifications=true showExperimentalOptions=true updateSegment=50 useNewBigFolderSizeLimit=true

Fixed Config

[General] clientVersion=3.6.4git (build 12826) confirmExternalStorage=true newBigFolderSizeLimit=500 optionalServerNotifications=true showCallNotifications=true showExperimentalOptions=true updateSegment=50 useNewBigFolderSizeLimit=true

maxChunkSize=999000000 minChunkSize=1000000 targetChunkUploadDuration=6000

ph818 commented 1 year ago

maxChunkSize=50000000

This worked for me in Windows 10, my Nextcloud desktop client is version 3.6.4 I am also using Cloudflare.

tomshomelab commented 1 year ago

where do you enter this data for AIO verion of nextcloud, my file is located in /var/lib/docker/volumes/nextcloud_aio_nextcloud/_data/config/config.php but i cannot find where to enter this on the document that somewhat matches

ph818 commented 1 year ago

I put mine in client config, not server config. Unless I misunderstood and your client is in a docker? If there is a server-side setting that asks for this mode, I do not know it.

On Wed, Jan 4, 2023 at 6:32 PM tomshomelab @.***> wrote:

where do you enter this data for AIO verion of nextcloud, my file is located in /var/lib/docker/volumes/nextcloud_aio_nextcloud/_data/config/config.php but i cannot find where to enter this on the document that somewhat matches

— Reply to this email directly, view it on GitHub https://github.com/nextcloud/desktop/issues/4278#issuecomment-1371545327, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARDYZJI4CSISE4IBGLED3ZTWQYCB3ANCNFSM5OJXM27Q . You are receiving this because you commented.Message ID: @.***>

Gorgeous-Patrick commented 1 year ago

maxChunkSize=50000000

Thank you so much. It works on my machine.

tomshomelab commented 1 year ago

where

I put mine in client config, not server config. Unless I misunderstood and your client is in a docker? If there is a server-side setting that asks for this mode, I do not know it. On Wed, Jan 4, 2023 at 6:32 PM tomshomelab @.> wrote: where do you enter this data for AIO verion of nextcloud, my file is located in /var/lib/docker/volumes/nextcloud_aio_nextcloud/_data/config/config.php but i cannot find where to enter this on the document that somewhat matches — Reply to this email directly, view it on GitHub <#4278 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARDYZJI4CSISE4IBGLED3ZTWQYCB3ANCNFSM5OJXM27Q . You are receiving this because you commented.Message ID: @.>

im not sure i understand - client is the desktop application isnt it? my nextcloud is hosted on a container in portainer that i connect to via my pc, phone etc.

where are you entering this command?

vithusel commented 1 year ago

where

I put mine in client config, not server config. Unless I misunderstood and your client is in a docker? If there is a server-side setting that asks for this mode, I do not know it. On Wed, Jan 4, 2023 at 6:32 PM tomshomelab @._> wrote: where do you enter this data for AIO verion of nextcloud, my file is located in /var/lib/docker/volumes/nextcloud_aio_nextcloud/data/config/config.php but i cannot find where to enter this on the document that somewhat matches — Reply to this email directly, view it on GitHub <#4278 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARDYZJI4CSISE4IBGLED3ZTWQYCB3ANCNFSM5OJXM27Q . You are receiving this because you commented.Message ID: @_._>

im not sure i understand - client is the desktop application isnt it? my nextcloud is hosted on a container in portainer that i connect to via my pc, phone etc.

where are you entering this command?

This config isn't for the server so no changes need to be made server side. It's for the client.

I've included the instructions that someone else posted. You will need to do this on every machine the NextCloud Desktop client is running on. (Does not include the mobile apps so IOS and Android is fine)

Windows Fix Press Win+R on your keyboard to open the Run application. Past the following in the dialog box: %APPDATA%\Nextcloud\nextcloud.cfg This will either ask you to pick an application to open nextcloud.cfg or will open in your default text editor (unless you have something else set to open .cfg files). If it asks you to pick an application, feel free to use Notepad or any other editor. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

MacOS Fix Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box: $HOME/Library/Preferences/Nextcloud Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

Linux Fix Open a terminal window and edit the following file: nano $HOME/.config/Nextcloud/nextcloud.cfg Add the following line under the [General] section: maxChunkSize=50000000 Save the file (Ctl+o, Ctl+x), then quit Nextcloud desktop, and start it again.

tomshomelab commented 1 year ago

Perfect! thanks for posting this and clarrifying client or server side.

pcking99 commented 1 year ago

When is this gonna be fixed permanently? It's inconvenient to change the config file on every computer/user. Imagine having maybe 150 clients.

My setup is running nextcloud via Snap and having nginx proxy manager to direct the internet to it.

HHUBSS commented 1 year ago

I agree with others. This should be server-side or maybe add it to nextcloud client by default. I'm using 3.6.6 client version btw.

GuillaumeHemmen commented 1 year ago

I have this issue on a freshly installed 3.6.6 client on macOS. The issue was solved after updating the config file for the client.

jospoortvliet commented 1 year ago

When is this gonna be fixed permanently? It's inconvenient to change the config file on every computer/user. Imagine having maybe 150 clients.

My setup is running nextcloud via Snap and having nginx proxy manager to direct the internet to it.

Well technically it's a work-around for a bad/limited network setup. Of course in reality, you can't always fix that where it should be fixed (in the network) so you tell the client to work around it. In an ideal world, the client would be smart enough to realize that the network is dropping chunks of >100 mb and apply the limit by itself, but this logic simply isn't there and somebody has to put in the time to develop that. No volunteer has stepped up for this yet, from what I can tell. Perhaps it comes up with support sometimes, but I suspect our engineers then direct the customer to fix their network, rather than writing this code - as that is a faster and easier fix and they are very busy.

So all in all, the answer is more or less "when somebody who is capable of fixing it is annoyed enough by it to fix it in the code of the client rather than just add these lines to their client configuration, or when a customer can't fix their network and the client has to be changed to work around it".

Sorry, I don't have a much better answer, though voting up this issue (giving it a thumbs up) might mean somebody who fixes issues by priority (and some of our devs do that sometimes) will fix it sooner.

But if I sort for issues by 👍🏻 it seems this issue is a long way off from being the most liked. This likely means not a lot of people have this problem.

vithusel commented 1 year ago

When is this gonna be fixed permanently? It's inconvenient to change the config file on every computer/user. Imagine having maybe 150 clients. My setup is running nextcloud via Snap and having nginx proxy manager to direct the internet to it.

Well technically it's a work-around for a bad/limited network setup. Of course in reality, you can't always fix that where it should be fixed (in the network) so you tell the client to work around it. In an ideal world, the client would be smart enough to realize that the network is dropping chunks of >100 mb and apply the limit by itself, but this logic simply isn't there and somebody has to put in the time to develop that. No volunteer has stepped up for this yet, from what I can tell. Perhaps it comes up with support sometimes, but I suspect our engineers then direct the customer to fix their network, rather than writing this code - as that is a faster and easier fix and they are very busy.

So all in all, the answer is more or less "when somebody who is capable of fixing it is annoyed enough by it to fix it in the code of the client rather than just add these lines to their client configuration, or when a customer can't fix their network and the client has to be changed to work around it".

Sorry, I don't have a much better answer, though voting up this issue (giving it a thumbs up) might mean somebody who fixes issues by priority (and some of our devs do that sometimes) will fix it sooner.

But if I sort for issues by 👍🏻 it seems this issue is a long way off from being the most liked. This likely means not a lot of people have this problem.

Just so you are aware,

PR https://github.com/nextcloud/desktop/pull/4826

is a fix for this. Although there doesn't seem to be much in terms of movement from maintainers to either apply or reject it.

If anyone needs it. I've created a batch script that automates adding the values to nextcloud.cfg. PM me or I can also post it here.

LokeYourC3PH commented 1 year ago

Was sceptical if this fix would work for me, but it indeed solved my issues as well. Web uploads are notoriously slow, as they stop every 20MB (chunks) for a couple of seconds, which the Desktop version doesn't do. However, ever since upgrading to NextCloud 25, changing hosts and a couple of other changes have been made, the Desktop version had refused to upload large files for more than a couple of seconds without to then drop the connection. Applying the config changes indeed made the uploads continous, so there seems to be an issue with the NextCloud Desktop client. To be fair however, I had the same issue when using WebDAV, so there must be more to it.

eenturk commented 1 year ago

Thanks adding maxchunksize to config solved the issue for now. Hopefully this will be the default in the next update.

mtthidoteu commented 1 year ago

this needs to be addressed!!!

MiguelNdeCarvalho commented 1 year ago

Hey, is there any updated on this? This should be fixed ASAP, as it is a common problem for many users

pcking99 commented 1 year ago

The issue is still not fixed. The temp fix with the config works but is annoying to add for each computer. Not very helpful for companies using nextcloud.

rodrigo-mahamud commented 1 year ago

Please Fix that ASAP guys

jeQuentY commented 1 year ago

As long as you don't have multiple computers connected to the same nextcloud it's fine, the workaround works perfectly. Hopefully it'll get fixed soon enough. Until then, this is our best option:

Figured I would add my experience

Problem "413" Error after uploading ~1GB into a 22GB file using desktop Version 3.6.4 (macOS) Error was NOT reproducible when uploading via web.

I have "experimental" virtual file support enabled. I use cloudflare for my DNS but do NOT use proxy. I use Let's Encrypt CA for SSL.

Upload was on LAN

Not Working Config

[General] clientVersion=3.6.4git (build 12826) confirmExternalStorage=true newBigFolderSizeLimit=500 optionalServerNotifications=true showCallNotifications=true showExperimentalOptions=true updateSegment=50 useNewBigFolderSizeLimit=true

Fixed Config

[General] clientVersion=3.6.4git (build 12826) confirmExternalStorage=true newBigFolderSizeLimit=500 optionalServerNotifications=true showCallNotifications=true showExperimentalOptions=true updateSegment=50 useNewBigFolderSizeLimit=true

maxChunkSize=999000000 minChunkSize=1000000 targetChunkUploadDuration=6000

fluffy-kaiju commented 1 year ago

clientVersion=3.8.1stable-Win64 (build 20230419) ... maxChunkSize=999000000 minChunkSize=1000000 targetChunkUploadDuration=6000

Work for me with a 14Gb download

wallflower1965 commented 1 year ago

where

I put mine in client config, not server config. Unless I misunderstood and your client is in a docker? If there is a server-side setting that asks for this mode, I do not know it. On Wed, Jan 4, 2023 at 6:32 PM tomshomelab @._> wrote: where do you enter this data for AIO verion of nextcloud, my file is located in /var/lib/docker/volumes/nextcloud_aio_nextcloud/data/config/config.php but i cannot find where to enter this on the document that somewhat matches — Reply to this email directly, view it on GitHub <#4278 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ARDYZJI4CSISE4IBGLED3ZTWQYCB3ANCNFSM5OJXM27Q . You are receiving this because you commented.Message ID: @_._>

im not sure i understand - client is the desktop application isnt it? my nextcloud is hosted on a container in portainer that i connect to via my pc, phone etc. where are you entering this command?

This config isn't for the server so no changes need to be made server side. It's for the client.

I've included the instructions that someone else posted. You will need to do this on every machine the NextCloud Desktop client is running on. (Does not include the mobile apps so IOS and Android is fine)

Windows Fix Press Win+R on your keyboard to open the Run application. Past the following in the dialog box: %APPDATA%\Nextcloud\nextcloud.cfg This will either ask you to pick an application to open nextcloud.cfg or will open in your default text editor (unless you have something else set to open .cfg files). If it asks you to pick an application, feel free to use Notepad or any other editor. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

MacOS Fix Open a Finder window and press Command+Shift+G on your keyboard. This will bring up a 'Go to folder' window. Paste the following in the dialog box: $HOME/Library/Preferences/Nextcloud Open the nextcloud.cfg file. If you do not have a default editor for .cfg files, feel free to open the file with TextEdit. Add the following line under the [General] section: maxChunkSize=50000000 Save the file, quit Nextcloud desktop, and start it again.

Linux Fix Open a terminal window and edit the following file: nano $HOME/.config/Nextcloud/nextcloud.cfg Add the following line under the [General] section: maxChunkSize=50000000 Save the file (Ctl+o, Ctl+x), then quit Nextcloud desktop, and start it again.

It works (Windows 10)! Thank you so much .-)