[BUG] File Manager Uploads Do Not Complete 100% each time. Upload progress bar flickers up and down progress.

thabaum commented 1 week ago

Oqtane Info

Version - 5.2.1 Render Mode - Static Interactivity - Server Database - SQL Server Environment - IIS Production

Describe the bug

File Manager Uploads sometimes do not complete (mp3 file) and the progress bar flickers up and down while uploading.

Sometimes the file does not upload 100%
The UI progress bar flickers
After upload completes it takes a long time for the UI to show the message indicating the file uploaded successfully.

Expected Behavior

The upload always is completed and the progress bar is smooth and accurate not bouncing up and down.

Steps To Reproduce

Create a site in IIS Production environment Upload file(s) in the admin dashboard file manager

Anything else?

sbwalker commented 1 week ago

@thabaum I cannot reproduce this issue and unless it is possible to replicate the problem it will be almost impossible to fix. Can you please provide more details such as the size of the file being uploaded, etc... (perhaps even share the actual file you are uploading).

Also can you please describe what you mean by "flickers" - the file upload breaks a file into chunks so that it can upload it using multiple browser threads - so for a large file the progress indicator displays the progress for each chunk - which means that you will see the progress indicator change from 0% to 100% many times as the chunks are uploaded. If you do not like this behavior you can set the property ShowProgress="false" in your module.

File uploading is multi-threaded and asynchronous so the client needs to poll the server to determine if the file upload has completed or not. It uses the following algorithm to get the size of the file and then assumes an average of 2 Mbps network upload speed to calculate the polling interval - which it then attempts 5 times. This means that if the file upload failed you need to wait the maximum duration (ie. 5 X polling interval) to receive a message that it failed. What it your upload speed on your network?

                        var size = Int64.Parse(uploads[upload].Split(':')[1]); // bytes
                        var megabits = (size / 1048576.0) * 8; // binary conversion
                        var uploadspeed = 2; // 2 Mbps (3G ranges from 300Kbps to 3Mbps)
                        var uploadtime = (megabits / uploadspeed); // seconds
                        var maxattempts = 5; // polling (minimum timeout duration will be 5 seconds)
                        var sleep = (int)Math.Ceiling(uploadtime / maxattempts) * 1000; // milliseconds

More info on file upload is in this blog: https://www.oqtane.org/blog/!/17/file-upload-in-blazor - which has been optimized significantly since it was introduced in 2019 but the fundamental architecture has remained the same. You will notice that file upload is a LOT more complex than most people imagine.

thabaum commented 6 days ago

I would expect this progress bar to use chunks to 100% but figure out how many chunks are to be uploaded and adjust the bar each time a chunk is uploaded.

It looks like maybe it is doing both? as the progress bar continues to grow. I have done this with both wave and MP3 files. So try uploading a song and it should reproduce the issue.

I can email you a link to the actual file which is a pre release of a song soon to hit stores so not able to share it here yet.

thabaum commented 6 days ago

@sbwalker If it shows how many bytes are left along with how many uploaded and moved smoothly according to this would be more like any other progress bar.

By the way you describe it is like 2 progress bars are working at the same time.

Watching it, i can see this is probably what is going on...

sbwalker commented 4 days ago

@thabaum #4606 should address the issues you mentioned...

total size of all files being uploaded is now calculated prior to initiating upload
progress is updated to reflect actual upload progress in relation to the total size
no more flickering of progress info
polling interval for determining completion now takes into consideration if you are running locally (ie. no latency)
API now returns NoContent(204) to avoid "XML Parsing Error: no root element found Location"

thabaum commented 4 days ago

@sbwalker awesome, thank you!

oqtane / oqtane.framework