search

good20087758521 / apjp

Automatically exported from code.google.com/p/apjp
0 stars 0 forks source link

Out of memory: unable to create new native thread when using https #54

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1.setting up APJP proxy in WIN7 64bit/java 1.6(32bit). the configuration files 
attached
2.using chrome browser 16.0.912.77.m with google search instant on
3.visit https://www.google.com
google search instant will send ajax query when you change the query text.
Please pay attention to the memory APJP consumes. It increases all along when 
HTTPS requests being sent.
After a couple of minutes, when the memory goes up to 300M. it cannot create 
threads any more

What is the expected output? What do you see instead?
APJP could work properly on HTTPS proxy for a long while.

What version of the product are you using? On what operating system?
APJP 0.90 on Win7

Please provide any additional information below.
Attach the configuration.
For HTTP proxy, it works better but it seems memory leak still happen.

Exception in thread "Thread-6108" java.lang.OutOfMemoryError: unable to cre
ew native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Unknown Source)
        at APJP.ProxyServerWorker.run(ProxyServerWorker.java:110)
        at java.lang.Thread.run(Unknown Source)
Exception in thread "Thread-6109" java.lang.OutOfMemoryError: unable to cre
ew native thread
        at java.lang.Thread.start0(Native Method)
        at java.lang.Thread.start(Unknown Source)
        at APJP.ProxyServerWorker.run(ProxyServerWorker.java:144)
        at java.lang.Thread.run(Unknown Source)

Original issue reported on code.google.com by ympla...@gmail.com on 5 Feb 2012 at 8:20

Attachments:

GoogleCodeExporter commented 9 years ago 
I have added a maximum of 30 threads per proxy server.
Can you test attached APJP.jar?

Original comment by jeroen.v...@gmail.com on 5 Feb 2012 at 8:56

Attachments:

GoogleCodeExporter commented 9 years ago 
I am afraid this version does not work with https site.
When i try to use this version to visit HTTPS website such as https://google.com
or https://facebook.com. it always got errors as follows.

By the way I am running the PHP script /ngnix+fastcgi+php5 in a http web 
server..

Attached the logs for:
---------------------------------------------------------------------
APJP_LOCAL_HTTPS_PROXY_SERVER: HTTPS_SERVER/START OK
APJP_LOCAL_HTTPS_SERVER: REQUEST: HEAD / HTTP/1.1
APJP_LOCAL_HTTP_SERVER: REQUEST: GET http://clients2.google.com/service/update2/
crx?os=win&arch=x86&prod=chrome&prodversion=16.0.912.77&x=id%3Dmimojjlkmoijpicak
mndhoigimigcmbb%26v%3D0.0.0.0%26uc&x=id%3Dnpdjjkjlcidkjlamlmmdelcjbcpdjocm%26v%3
D0.0.0.0%26uc&x=id%3Dimmdilkhigodmjbnngapbehchmihabbg%26v%3D11.1.102.55%26uc HTT
P/1.1
APJP_LOCAL_HTTP_SERVER: RESPONSE: HTTP/1.0 200 OK
APJP_LOCAL_HTTP_SERVER: EXCEPTION
APJP.HTTP11.HTTPRequestException: HTTP_REQUEST/OPEN
        at APJP.HTTP11.HTTPRequest.open(Unknown Source)
        at APJP.HTTP.HTTPServerWorker.run(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source
)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
Caused by: java.net.ConnectException: Connection timed out: connect
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(Unknown Source)
        at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
        at java.net.PlainSocketImpl.connect(Unknown Source)
        at java.net.SocksSocketImpl.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at java.net.Socket.connect(Unknown Source)
        at sun.net.NetworkClient.doConnect(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.openServer(Unknown Source)
        at sun.net.www.http.HttpClient.<init>(Unknown Source)
        at sun.net.www.http.HttpClient.New(Unknown Source)
        at sun.net.www.http.HttpClient.New(Unknown Source)
        at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown
Source)
        at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Sour
ce)
        at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
        ... 5 more

Original comment by ympla...@gmail.com on 5 Feb 2012 at 9:58

GoogleCodeExporter commented 9 years ago 
Sorry.  I have made a mistake to test latest version. 
The latest test result:
The HTTPS proxy server stop working after a few requests from 
https://www.google.com. It seems its connections always timeout after that. 
There is no output from the console. But from browser i could not connect any 
HTTPS server. While HTTP web site  works fine.
After restart it could accept a few of connection again but after a few of 
request it is down again,too.

Original comment by ympla...@gmail.com on 5 Feb 2012 at 10:37

GoogleCodeExporter commented 9 years ago 
I am testing with CHROME 16.0.912.77 m, JAVA 1.6.0_26, WINDOWS 7 32bit and 
APJP_LOCAL-0.9.0 + your APJP.properties.
I go to https://www.google.com and click IMAGES, then I search something.
Everything works, no OUT-OF-MEMORY or CONNECTION-TIMED-OUT problems.

Do you have the same problem with another browser?

Original comment by jeroen.v...@gmail.com on 5 Feb 2012 at 10:49

GoogleCodeExporter commented 9 years ago 
Do you have the same problem with APJP_LOCAL + APJP_REMOTE_APPENGINE?

Original comment by jeroen.v...@gmail.com on 5 Feb 2012 at 10:54

GoogleCodeExporter commented 9 years ago 
Do you have the same problem with APJP_LOCAL-0.8.9?

Original comment by jeroen.v...@gmail.com on 5 Feb 2012 at 11:45

GoogleCodeExporter commented 9 years ago 
I would like to close this issues because I think it's enough for personal use 
already.But I do not know how to close it. 

As a matter of fact what I did belongs to a kind of extreme condition.----I 
keep pressing the keyboard in google search box to let it send large amounts 
the AJAX requests.To think it as a performance test. Firstly I encountered the 
issue:"address in use" and the HTTPS proxy refuse to work.-----The OS had not 
recycled the socket resource in time yet.

After that I had changed some code to let software to bind more sockets. from 
100 to 500. I continued to do the performance test. I believed that OS does not 
release some
memory after each request. But if i surf the internet at normal speed, I hardly 
notice the memory issue.  So I think it's fare to accept it.

Thanks for quick reply. Your team have make great things..

Do you have the same problem with another browser? ----  Yes
Do you have the same problem with APJP_LOCAL + APJP_REMOTE_APPENGINE? --- not 
test
Do you have the same problem with APJP_LOCAL-0.8.9? ---not test. if you send 
the file i will do it.

Original comment by ympla...@gmail.com on 5 Feb 2012 at 1:17

GoogleCodeExporter commented 9 years ago 
I understand the problem now :)

For each HTTPS request:
APJP_LOCAL_PROXY_SERVER forwards the HTTPS request to the 
APJP_LOCAL_HTTPS_PROXY_SERVER
APJP_LOCAL_HTTPS_PROXY_SERVER starts the APJP_LOCAL_HTTPS_SERVER for the HTTP 
request
APJP_LOCAL_HTTPS_PROXY_SERVER forwards the HTTPS request to the 
APJP_LOCAL_HTTPS_SERVER
APJP_LOCAL_HTTPS_SERVER forwards the HTTPS request to the 
APJP_REMOTE_HTTPS_SERVER
APJP_LOCAL_HTTPS_PROXY_SERVER stops the APJP_LOCAL_HTTPS_SERVER for the HTTP 
request

As you can see a lot of sockets need to be opened/closed for a HTTPS request.
When a socket is closed, the OS waits 2MSL before the socket can be reopened 
(TIME_WAIT).

APJP_LOCAL_PROXY_SERVER just forwards HTTP/HTTPS requests, so you can configure 
your browser with APJP_LOCAL_HTTP_PROXY_SERVER and 
APJP_LOCAL_HTTPS_PROXY_SERVER (better performance).

When you send a lot of HTTPS requests, you can overload the server where  
APJP_REMOTE_HTTPS_SERVER is installed (CONNECTION-TIMED-OUT).
You should install APJP_REMOTE_HTTPS_SERVER on a several servers and configure 
APJP_LOCAL with those servers (load balancing). You can add up to 10 
APJP_REMOTE_HTTPS_SERVER servers.

You can test load balancing with APJP_REMOTE_APPENGINE.

Original comment by jeroen.v...@gmail.com on 5 Feb 2012 at 2:19

GoogleCodeExporter commented 9 years ago 
Thanks! Now I understand more. Let me try it.

Original comment by ympla...@gmail.com on 6 Feb 2012 at 6:20

GoogleCodeExporter commented 9 years ago 
I think there is a bug in APJP_REMOTE_HTTPS_SERVER load balancing :(
I will correct that bug in APJP_LOCAL-0.9.1

Original comment by jeroen.v...@gmail.com on 6 Feb 2012 at 6:40

GoogleCodeExporter commented 9 years ago 
You should try APJP_LOCAL-0.9.3 (APJP_LOCAL-0.9.3 supports HTTP/1.1 persistent 
connections)

Original comment by jeeto.je...@gmail.com on 9 Mar 2012 at 6:03