oqtane / oqtane.framework

CMS & Application Framework for Blazor & .NET MAUI
http://www.oqtane.org
MIT License
1.89k stars 547 forks source link

[BUG] Module Management : Upload a large module (bigger than 1GB) results in an unusable installation file. #4795

Closed HQuast closed 2 weeks ago

HQuast commented 3 weeks ago

Oqtane Info

Version - 5.2.4 Render Mode - Static Interactivity - Server Database - Postgres

Describe the bug

Uploading a module package that is larger than 1 Gb results in a an unusable installation file (nupkg). That happened when running in a Docker container. The reason is that the the chunked files are merge in the order they are stored. But they should be merged in the ordered by the file naming.

Expected Behavior

Steps To Reproduce

Anything else?

Solution: File :oqtane.framework\Oqtane.Server\Controllers\FileController.cs Commit: #3adb7ecb1cabcc579b68350e51061299f86b20e9

Class : public class FileController : Controller Method: private async Task MergeFile(string folder, string filename)

LIne: 520 Replace the line
foreach (string filepart in fileparts) With foreach (string filepart in fileparts.Order())

sbwalker commented 3 weeks ago

@HQuast the file upload process breaks a file into 1 MB chunks and uploads them using multiple browser threads. It stores the chunks on the server with a file name extension of ".part######" (ie. ".part_001_999"). Once all parts have been uploaded it merges them back into a single file. The issue you are experiencing is that a file which is over 1 GB has more than 999 parts - which means that it exceeds the file naming convention described above. As a result, the file upload will not be successful. Can you please explain why you need to upload files which are greater than 1 GB to your installation?

HQuast commented 3 weeks ago

@sbwalker Sorry, I mean MB. In my test the upload was splitted in two file chunks. And the files are merged in reverse order. I could only reproduce this when running in a docker container. On a local installation the problem didn't occurs.

sbwalker commented 3 weeks ago

Ok, so you are saying the call to:

Directory.GetFiles(folder, filename + token + "*");

is resulting in a list of files which is not in alphabetical order when run in Docker... very strange.

sbwalker commented 3 weeks ago

In the documentation:

https://learn.microsoft.com/en-us/dotnet/api/system.io.directory.getfiles?view=net-8.0

It actually says:

"The order of the returned file names is not guaranteed; use the Sort method if a specific sort order is required."

Is the Docker container running Windows or Linux?

HQuast commented 3 weeks ago

It is a Linux container.