Allow chunked public upload

[tessus]

I've noticed that for the file drop or any other screen that allows one to upload a file, there's no way to know in advance if the upload will fail.

Let's say the max upload size is 100 MB and someone tries to upload 105 MB. The upload will fail, but the person still uploaded 100MB for nothing. It would be nice, if there was a way to handle this scenario. The easiest would be to show a message that tells an uploader the max upload size.

What do you think?

[MorrisJobke]

We plan to also do chunking on the web UI upload which would mean that we don't have a limit anymore.

Sadly I can't find the issue for this - @rullzer @jancborchardt @ChristophWurst Could you find it or was there none, yet?

[tessus]

Sounds good, but I suspect that implementing this chunking feature will take a while (months, years).

In the meantime a message would be great and very helpful. Adding the message won't be much of an effort for someone who knows the code.

[lesykm]

May be this could be reused: https://github.com/owncloud/core/pull/28415/files

[MorrisJobke]

Web UI does chunking since Nextcloud 13 - we ported exactly this pr in #7056. Don't know if files drop and unauthenticated is also supported as of now. cc @danxuliu

[aryasenna]

Any new update to this? Any chance this will come on next major release?

[jay-tuckey]

This feature would be very handy for the university I work for, as we would like to be able to use the "file drop (upload only)" feature for student work submission, including videos projects which can be very large, ~2GiB. Currently we are testing a workaround by just increasing the max upload size, but then the student has no progress visible on how their upload is going or whether it's working. Any progress on this feature for public uploads?

[ChristophWurst]

@jay-tuckey do you have an Enterprise support subscription? Just asking because then we might be able to work on this in a professional services project.

[jay-tuckey]

@christophWurst unfortunately not at the moment, we may get a subscription in the future if we end up using nextcloud for more areas.

[tessus]

3 years later... Basic core functions are apparently not a priority.

[jospoortvliet]

@tessus what is basic core functionality is in the eye of the beholder. As both private AND enterprise user I've never needed this. There is an easy work-around (just create a guest account) so I'm glad the team focuses on more important things.

[X-Coder]

I just wasted some hours figuring why the upload works everywhere but not on public sharing until I have seen it has to do with missing chunked file upload.

Could you please at least add a hint in the official docs about non available chunk-uploading for public sharing if you don't plan to fix this?

[FlashyDuck]

This is also affects WebDAV connections.

When using Cloudflare proxy uploads larger than 100MB fail. Chunking apparently needs to be forced by the user, but how?

[djdarkstep]

Yes, this bug totally breaks public uploads when using Cloudflare as proxy. When uploading from a public upload-enabled link, the entire file is sent in one single PUT request: /public.php/webdav/filename.ext

The free Cloudflare plan limits HTTP POST request size to 100MB, so if you try to publicly upload a file that is larger than 100MB, the upload fails.

[Rick45]

I have a scenario where i need to upload files up to 5G from a shared folder, as workaround i have added client_max_body_size 5G; in nextcloud/config/nginx/site-confs file

[911realdeb]

@Rick45 > I have a scenario where i need to upload files up to 5G from a shared folder, as workaround i have added client_max_body_size 5G; in nextcloud/config/nginx/site-confs file

THANKS ! it work now !!!!

[Lynxiayel]

I have a scenario where i need to upload files up to 5G from a shared folder, as workaround i have added client_max_body_size 5G; in nextcloud/config/nginx/site-confs file

I've run into same problem, but cannot find nextcloud/config/nginx/site-confs, there's only config.php in the config folder, can I have some tips? thanks.

[nooblag]

Try looking somewhere like /etc/nginx/sites-available/ if you're using nginx.

However, sadly, it doesn't seem to be a fix anyway. I tried client_max_body_size 5G; and client_max_body_size 0; and numerous other large and small values (restarting nginx after each change/each time) and still, nothing over 100MB uploads successfully.

[Nirostar]

Try looking somewhere like /etc/nginx/sites-available/ if you're using nginx.

However, sadly, it doesn't seem to be a fix anyway. I tried client_max_body_size 5G; and client_max_body_size 0; and numerous other large and small values (restarting nginx after each change/each time) and still, nothing over 100MB uploads successfully.

Do you use cloudflare or some other proxy? Cloudflare does limit the upload to 100MB :/

[alfonsrv]

Shouldn't use GitHub issues for 101 tech support.

Enabling a bigger client body in nginx/apache2 definitely fixes the undesired behaviour for public links.

[nooblag]

Yes, using Cloudflare---hence the desire for chunking to be implemented to fix the problem, like it is for logged-in users.

[Nirostar]

Shouldn't use GitHub issues for 101 tech support.

Enabling a bigger client body in nginx/apache2 definitely fixes the undesired behaviour for public links.

Yeah but there are a lot of scenarios where you can't influence the max client body (Proxies, hosted systems etc.) Implementing chunked uploads for anonymous users would solve this instantly.

[Slopz]

Anyone found any workarounds? I really don't want to get rid of Cloudflare...and I also don't want to create user logins for every share link I send...

[nooblag]

Sadly, no. I hope the developers add file chunking to public uploads (just as is available for logged in users) sometime soon. 🤞 We do not want to get rid of Cloudflare either.

[HerbertHorst]

Is there any update on this? Im on version "Nextcloud Hub 4 26.0.2" and ran into this problem last week. Took me a few days to find what the problem is, but didnt found a solution yet.

[realies]

Chunking in publicly shared folders still does not work.

[ufo56]

Any news ?

[joshtrichards]

Historical context in case you feel like making an attempt at prototyping an implementation:

https://github.com/owncloud/core/issues/8955#issuecomment-345206696

https://github.com/owncloud/core/pull/28415#issuecomment-344632599

And raw protocol details for chunking:

https://docs.nextcloud.com/server/latest/developer_manual/client_apis/WebDAV/chunking.html

[joshtrichards]

30069 may open the door to this functionality being a bit easier to implement from the looks of it

[austin3410]

Verified this is still an issue on 27.1.4.... at least 6 years later.

[joshtrichards]

Well, it's an enhancement idea not a bug, and this is an open source project. The issue is still open, which indicates no one has jumped in and completed (volunteered) to do the work. You're welcome make an attempt at implementing it!

[ShaunCurrier]

Is there a feature bounty program for Nextcloud? I'd love to throw some money that way for this issue.

[austin3410]

Well, it's an enhancement idea not a bug, and this is an open source project. The issue is still open, which indicates no one has jumped in and completed (volunteered) to do the work. You're welcome make an attempt at implementing it!

Without any sort of warning, or in-app acknowledgement that File Drop links are meant for tiny files and anything that may be a "large" file may not work, I would still consider it a bug or at the very least a poorly implemented feature.

I had to spend an entire day searching through logs to finally figure out that the upload methods for authenticated vs unauthenticated sessions are completely different. If there was at least some sort of warning explaining the handicaps of using File Drop links I would've reconsidered my approach instead of wasting time searching for a problem on my end that wasn't there. ¯_(ツ)_/¯

*end of rant lol

[alfonsrv]

You got some free extra knowledge on how the web works through troubleshooting it though 💭

[joshtrichards]

Fair enough.

I don't recall offhand how/if it's noted across the Admin and User manuals. You're welcome to suggest some refined or added language to the appropriate section(s) in one or both manuals by - https://docs.nextcloud.com. There is an edit button in the upper right.

Unfortunately NC itself has no way of knowing the limit because it's environment dependent and determined by the end to end configuration of app servers, web servers and proxies. Technically it's possible to upload fairly large files, even without without chunking, with the appropriate tuning as noted elsewhere in this Issue (and in the >512M Uploads section of the Admin Manual).

But, yes, a note somewhere makes sense if there isn't one.

[austin3410]

You got some free extra knowledge on how the web works through troubleshooting it though 💭

lol fair XD

[nooblag]

Great discussion around this.

I only have this to add:

Unfortunately NC itself has no way of knowing the limit because it's environment dependent [...]

That part (at the server end where Nextcloud is running) is discoverable through ini_get('upload_max_filesize') at least...

Yes there would be a challenge detecting with anything much beyond that (i.e. proxies and so on) but that's our point---that all of this would be fixed (made redundant) with a chunked file upload. :)

[TimKraemer]

I control everything from metal to nextcloud, which would be debian -> docker -> nginx proxy -> php -> nextcloud but I still can't manage to allow guests without registration to upload huge files to file drop - did anyone succeed with a setup like mine?

[realies]

I control everything from metal to nextcloud, which would be debian -> docker -> nginx proxy -> php -> nextcloud but I still can't manage to allow guests without registration to upload huge files to file drop - did anyone succeed with a setup like mine?

Same here, and not yet. Maybe in another six years.

[llek1000]

Hello, any updates on this?

[EuleMitKeule]

+1 on this.

[jospoortvliet]

For those waiting for this - the public file view is now also nearly finished with its Vue migration, which will allow enabling chunked upload. I'm not sure it will make it in Hub 9 (September), or just miss the deadline and come in Hub 10 (early next year), but it seems very close. Personally also looking forward to it, the new vue-ified public link shares will let us focus on some other nice sharing related features.

[realies]

omfg 7 years and it's finally complete 🍾

[Slopz]

What a time to be alive.

[szaimen]

🎉