search

davideuler / gss

Automatically exported from code.google.com/p/gss
Other
0 stars 0 forks source link

unable to upload big files #93

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
What steps will reproduce the problem?
1. Use firefox 4.0.1 (on windows xp/7 or linux) or Internet Explorer 8 and try 
to upload big file - I tested > 4gb files
2. In server log file there will be wrong content-length and GSS IO exception 
will occur
3. File upload dialog progress bar is frozen and no file is actually uploaded 
since user gets HTTP 500 response (because of I believe wrong content-length).

What is the expected output? What do you see instead?

Expected is file being uploaded, and instead file upload dialog and progress 
bar looks frozen, nothing happens. Errors show up in server.log file (check 
attached files). 

What version of the product are you using? On what operating system?

We are using gss built from source dated 30/11/2010 and 09/03/2011 on Centos 5 
64bit and same issue happens. Tested on windows 7 64bit with firefox 4.0.1 and 
windows xp sp3 with firefox 4.0.1. 

Please provide any additional information below.

With smaller files (e.g. 650mb cd isos) content length is correct and upload 
works fine on all tested client machines. 

Regards,
Nikola

Original issue reported on code.google.com by ngara...@gmail.com on 4 May 2011 at 1:51

Attachments:

GoogleCodeExporter commented 8 years ago 
Also I noticed today with windows xp uploading 6.6gb file results in 514byte 
file that is unusable. It seems like file is being deleted, since its size is 
obviously very wrong. User did not delete file himself :)

From logs:

Starting upload:

05 May 2011 10:56:28,468 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] [POST] 
/user@srce.hr/files/cbgb-devil1080.mkv
05 May 2011 10:56:28,469 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] content-type: application/octet-stream
05 May 2011 10:56:28,469 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] X-GSS-Date: Thu, 5 May 2011 08:56:58 UTC
05 May 2011 10:56:28,469 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] authorization: user@srce.hr 
Nem00pgc5CCSSgmPYztlnLCewM4=
05 May 2011 10:56:28,469 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] accept: application/json; charset=utf-8
05 May 2011 10:56:28,470 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] content-length: 7040534772
05 May 2011 10:56:28,470 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] Accept-Encoding: gzip, deflate
05 May 2011 10:56:28,470 ajp-0.0.0.0-8009-1 DEBUG () 
[gr.ebs.gss.server.rest.RequestHandler] user-agent: Mozilla/4.0 (compatible; 
MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 1.1.4322; 
InfoPath.2; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; AskTbUT2V5/5.9.1.14019)

After uploading:

05 May 2011 11:06:07,288 ajp-0.0.0.0-8009-11 DEBUG () 
[gr.ebs.gss.server.rest.FilesHandler] Serving resource '/cbgb-devil1080.mkv' 
headers only
05 May 2011 11:06:07,335 ajp-0.0.0.0-8009-11 DEBUG () 
[gr.ebs.gss.server.rest.FilesHandler] contentType='application/octet-stream'
05 May 2011 11:06:07,335 ajp-0.0.0.0-8009-11 DEBUG () 
[gr.ebs.gss.server.rest.FilesHandler] contentLength=514

Original comment by ngara...@gmail.com on 5 May 2011 at 11:13

GoogleCodeExporter commented 8 years ago 
Thanks for the report Nikola, we will check it out.

Original comment by fstamate...@gmail.com on 5 May 2011 at 2:04

GoogleCodeExporter commented 8 years ago 
So far I can say that browsers that do POST request when uploading (Firefox 
4.0.1 and IE8) fail with bigger files. Scenario I tested was files over 4gb in 
Windows 7 64bit, but it turns out every upload over 2gb causes this issue (if 
POST request is used for uploading, so it happens on Linux with FF 4.0.1 too). 
With files over 2GB server responds with HTTP 400 (bad request) and with files 
over 4GB server responds with HTTP 500 (internal server error).

So, if POST request is used, upload will fail with HTTP 400 if over 2GB and 
HTTP 500 if over 4GB.

Only weird scenario was uploading 6.6GB file with IE8 from windows xp and file 
magically appearing as 514byte in size instead of 6.6GB. I do not understand 
how and why, but I thought it is related to POST request and uploading of 
bigger files, so I added that to this issue earlier.

I do not use google gears, since I am running single upload at a time. Hope 
that is ok since I think it does not require google gears for bigger files. Am 
I wrong?

So, clients in test scenarios were: Windows 7 64bit, Windows xp sp3 (32bit) and 
Arch Linux 64bit.
GSS server runs on Centos 5 Linux 64bit.

Hope i did not miss out some parts of information.

Regards,
Nikola

Original comment by ngara...@gmail.com on 5 May 2011 at 5:10

GoogleCodeExporter commented 8 years ago 
Thank you for the detailed additional info, they are very helpful. We have 
recently identified a potential bug in POST uploads that seems to be related.

Original comment by fstamate...@gmail.com on 6 May 2011 at 10:29

GoogleCodeExporter commented 8 years ago 
After testing some scenarios I end up with the same results that are mentioned 
above.
Scenarios that are tested:

Scenario 1 - success
Title: Upload a file of 4.2 GB
Browser: Google Chrome
OS: Windows Vista Home 32bit
File Size: 4.2 GB
Method used in request: PUT ([org.gss_project.gss.server.rest.RequestHandler] 
[PUT] /lakis@ebs.gr/files/VIDEO_TS.zip)
Content-length: accurate ([org.gss_project.gss.server.rest.RequestHandler] 
content-length: 4504288722)

Scenario 2 - fail
Title: Upload a file of 4.2 GB
Browser: Firefox 4
OS: Windows Vista Home 32bit
File Size: 4.2 GB
Method used in request: POST ([org.gss_project.gss.server.rest.RequestHandler] 
[POST] /lakis@ebs.gr/files/file+4.2gb.zip)
Content-length: inaccurate ([org.gss_project.gss.server.rest.RequestHandler] 
content-length: 474
[org.gss_project.gss.server.rest.RequestHandler] content-length: 478
[org.gss_project.gss.server.rest.RequestHandler] content-length: 8670)
Database entry for file upload status shows:
bytesuploaded: 8670;
filesize: 8670;

Scenario 3 - fail
Title: Upload a file of 2.63 GB after the failure of the uploading of a file of 
4.2 GB
Browser: Firefox 4
OS: Windows Vista Home 32bit
File Size: 2.63 GB
The server responds with HTTP 404 respond in the request
http://localhost:8080/pithos/rest/lakis@ebs.gr/files/filename.zip?progress=filen
ame.zip

I couldn't reproduce HTTP 400 or HTTP 500 in firefox for either files with file 
size greater than 2 GB or 4 GB

Scenario 4 - fail
Title: Upload a file of 4.2 GB 
Browser: IE 8
OS: Windows Vista Home 32bit
File Size: 4.2 GB
Method used in request: POST ([org.gss_project.gss.server.rest.RequestHandler] 
[POST] /lakis@ebs.gr/files/file+4.2gb.zip)
Content-length: inaccurate ([org.gss_project.gss.server.rest.RequestHandler] 
content-length: 209321901 
Database entry for file upload status shows:
bytesuploaded: 209321901;
filesize: 209321901;

The scenario for uploading a file with the file size greater than 2 GB also 
fails in IE 8

Original comment by akaprave...@gmail.com on 11 May 2011 at 4:41