Huge file upload problem at 99%

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Upload a 697mb file or more.
2. Wait til it reaches 99%
3.

What is the expected output? What do you see instead?
It should do a complete 100% upload but it gets stuck to 99% with a blinking 
tip of the progress bar and it stays that way forever.

What version of the product are you using? On what operating system?
I just downloaded the source on this version and made and overwrite of all the 
previous version retaining the database info.  I was actually encountering the 
same problem before I downloaded this 1.5.1 version hoping that it would fix 
the huge file upload problem.

Please provide any additional information below.
Tested this with the old version prior to using 1.5.1 and it had the same 
problem with huge files.  Uploading small files did not seem to bother it but I 
tried a file as big as 697mb and it just gets stuck to 99% and never completes. 
 I also tested this on different browsers, chrome, ie, mozilla ff even on 
different platforms Win7 and Ubuntu it behaves the same way.

I do not know if this is a timeout issue on the webserver if yes, I would 
appreciate some help as to what settings would best make this work.  Thanks.

~rocky

Original issue reported on code.google.com by rcastan...@smartermanager.com on 18 Jul 2011 at 2:55

[GoogleCodeExporter]

Has anyone encountered or have a solution to this problem?

Original comment by rockycas...@gmail.com on 20 Jul 2011 at 8:19

[GoogleCodeExporter]

Not sure what's up with that, Rocky... are you uploading across the internet 
(ie, is there a possibility it's an ISP problem)?

I've successfully uploaded 50G files across a LAN.

Original comment by dobr...@gwainc.net on 31 Jul 2011 at 8:43

[GoogleCodeExporter]

OK, I see the issue - after the file has fully uploaded, the server actually 
COPIES the file from the temporary upload directory to the filestore, rather 
than moving it.  If you're uploading a really big file on a machine that 
doesn't have a lot of disk I/O available, this will produce a very noticeable 
lag at 99% while the file is being moved from the temp directory to the actual 
filestore.

This is clearly less than ideal.  Let me bash it around for a bit.

Original comment by jim.salt...@gmail.com on 31 Jul 2011 at 9:08

[GoogleCodeExporter]

Original comment by jim.salt...@gmail.com on 31 Jul 2011 at 9:09

• Changed state: Verified

[GoogleCodeExporter]

Original comment by jim.salt...@gmail.com on 31 Jul 2011 at 9:12

• Changed state: Accepted

[GoogleCodeExporter]

If there's a dev out there who feels like tackling this one, I'll be happy to 
review any patches you come up with.

Due to the way Relay handles multiple file uploads, there doesn't seem to be 
any real easy way to fix this.  Basically multiple file uploads are sent up as 
one continuous binary transmission with a content-type used as a boundary; once 
an upload is complete the upload.pl script parses through the whole thing a 
line at a time looking for the boundaries to split the one big binary back into 
multiple files.

If I'm reading the code correctly, this also means Relay will be RAM-limited in 
how large a file it can handle; the actual data transmission is serial and not 
limited by RAM but that nasty line-by-line parsing after the upload finishes 
tries to pull the whole transmission into memory at once.  Not good.

Original comment by jim.salt...@gmail.com on 31 Jul 2011 at 9:15

[GoogleCodeExporter]

Sorry, had to move to another FTP client.  Can't wait for the update on this.

Original comment by rockycas...@gmail.com on 4 Aug 2011 at 6:21

[GoogleCodeExporter]

hello, could it be possible to implement a solution like plupload to solve 
problemes like that ?

Original comment by noel.fro...@gmail.com on 4 Feb 2013 at 1:35