search

saokar / urlrewritefilter

Automatically exported from code.google.com/p/urlrewritefilter
Other
0 stars 0 forks source link

Improvement: When proxying requests - stream the data to reduce memory consumption #53

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?

1) Use urlrewritefilter to proxy requests to a new url
2) post large files to that url which are proxied over to other url
3) instead of streaming, the posted data is temporarily stored in memory 
   before proxied over to the other URL, causing out of memory exceptions

What is the expected output? What do you see instead?

Ability to upload (post) file with unlimited file sizes without increasing
the memory consumption of the server.

What version of the product are you using? On what operating system?

3.2.0

Please provide any additional information below.

Attached, please find a patch against the 3.2.0-tagged version from SVN
that fixed the issue for the project I am working on.

With the patch I am able to upload e.g. 1GB of data without increasing the
memory footprint of the servlet container. 

I implemented a custom request entity
(org.apache.commons.httpclient.methods.RequestEntity) that allows me to
directly stream the data.

Original issue reported on code.google.com by ghillert on 25 Sep 2009 at 4:28

Attachments:

GoogleCodeExporter commented 9 years ago 
The idea is good and the problem very legitimate. Will review the patch 
shortly. 
Thanks ghillert!

Original comment by avl...@gmail.com on 25 Sep 2009 at 4:48

GoogleCodeExporter commented 9 years ago 
Thanks for the fast response! Right now the streaming functionality is 
hard-coded -
Not sure whether it would be better to have both options (streaming or buffered)
(particular for request streaming.) 

Some information at: http://hc.apache.org/httpclient-3.x/performance.html

Let me know if I can be of further help.  

Thanks!

Gunnar

Original comment by ghillert on 26 Sep 2009 at 3:08

GoogleCodeExporter commented 9 years ago 
ghillert, if you can please add some test cases to your patch, it would make 
life
easy for me to apply the patch.

Original comment by avl...@gmail.com on 7 Jan 2010 at 5:32

GoogleCodeExporter commented 9 years ago 
Patch applied in 4.0.1.  Please verify.

Original comment by p...@tuckey.org on 25 Jun 2012 at 2:00