search

eBay / parallec

Fast Parallel Async HTTP/SSH/TCP/UDP/Ping Client Java Library. Aggregate 100,000 APIs & send anywhere in 20 lines of code. Ping/HTTP Calls 8000 servers in 12 seconds. (Akka) www.parallec.io
Apache License 2.0
805 stars 173 forks source link

Add <scope>test</scope> to test-only dependencies. #7

Closed kryger closed 9 years ago

jeffpeiyt commented 9 years ago

Thanks Lukasz for your pull request!

Most tests passed but only 1 test related to variable replacement in http header failed. I will check more on this why it is failing after the change. Appreciate your guidance too.

https://travis-ci.org/eBay/parallec/builds/90138531

Tests in error: 
  differentRequestsToSameTargetHostWithHeaderReplacement(io.parallec.core.main.http.request.template.ParallelClientVarReplacementHostSpecificHeaderTest)
jeffpeiyt commented 9 years ago

Look at the diff it should not relate to your changes... However I just run the tests locally and it passed. Let me check more too.

jeffpeiyt commented 9 years ago

Looks like the nano httpd server fail to start? and then the connection exception.

Could you please try to run the whole test suites with this change locally, and see if you are able to reproduce the issue Lukasz?

Thanks!

17:15:32.131 [ParallecActorSystem-akka.actor.default-dispatcher-5] INFO  i.p.c.TestBase - SingleTaskResponse [request=TaskRequest [actorMaxOperationTimeoutSec=15, resourcePath=/testHeaders, requestContent=, httpMethod=GET, pollable=false, httpHeaderMap={sample=222}, protocol=http, host=API_1, hostUniform=localhost, port=10080, sshMeta=null], responseContent=null, receiveTime=2015.11.09.17.15.32.127+0000, receiveTimeInManager=2015.11.09.17.15.32.128+0000, error=true, errorMessage=java.net.ConnectException, stackTrace=java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:976)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851)
    at com.ning.http.client.providers.jdk.JDKAsyncHttpProvider$AsyncHttpUrlConnection.call(JDKAsyncHttpProvider.java:245)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
, statusCode=NA, operationTimeMillis=9, pollingHistoryMap={}]
17:15:32.131 [ParallecActorSystem-akka.actor.default-dispatcher-6] INFO  i.p.c.a.ExecutionManager - 
[2]__RESP_RECV_IN_MGR 2 (+0) / 2 (100.00%)  AFT 0.018 S @ API_0 @ 2015.11.09.17.15.32.131+0000 , TaskID : d48495c6-491 , CODE: NA, RESP_BRIEF: EMPTY , ERR: java.net.ConnectException
17:15:32.132 [ParallecActorSystem-akka.actor.default-dispatcher-6] INFO  i.p.c.TestBase - SingleTaskResponse [request=TaskRequest [actorMaxOperationTimeoutSec=15, resourcePath=/testHeaders, requestContent=, httpMethod=GET, pollable=false, httpHeaderMap={sample=111}, protocol=http, host=API_0, hostUniform=localhost, port=10080, sshMeta=null], responseContent=null, receiveTime=2015.11.09.17.15.32.130+0000, receiveTimeInManager=2015.11.09.17.15.32.131+0000, error=true, errorMessage=java.net.ConnectException, stackTrace=java.net.ConnectException: Connection refused
    at java.net.PlainSocketImpl.socketConnect(Native Method)
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
    at java.net.Socket.connect(Socket.java:579)
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:211)
    at sun.net.www.http.HttpClient.New(HttpClient.java:308)
    at sun.net.www.http.HttpClient.New(HttpClient.java:326)
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:997)
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:976)
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:851)
    at com.ning.http.client.providers.jdk.JDKAsyncHttpProvider$AsyncHttpUrlConnection.call(JDKAsyncHttpProvider.java:245)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
, statusCode=NA, operationTimeMillis=11, pollingHistoryMap={}]
kryger commented 9 years ago

Hmm, I've just realised that running mvn test locally (on the current state of master, so excluding my modifications) fails for me locally:

Results :

Failed tests: sshWorkerFakeVmPasswordTest(io.parallec.core.main.ssh.ParallelClientSshTest): fail. error is not expected. not sure if ssh flow was executed

Tests run: 102, Failures: 1, Errors: 0, Skipped: 0

The same test fails when executed directly in Eclipse. The test output log shows a more detailed reason for failure (not sure if it isn't a red herring as it appears more than once in the full test output):

ERROR i.p.c.c.w.s.SshProvider - error in exec SSH. If exection is JSchException: Auth cancel and using public key. Make sure 1. private key full path is right (try sshMeta.getPrivKeyAbsPath()).

  1. the user name and key matches com.jcraft.jsch.JSchException: Auth fail

The test case failing in Travis (differentRequestsToSameTargetHostWithHeaderReplacement) passes both in Maven and Eclipse locally.

Is my local setup to blame? This would be kind of worrying as ideally the test suite should just pass out of the box (and it does pass on Travis, presumably with no extra initial setup). I'm on OSX if that makes any difference.

kryger commented 9 years ago

Strange thing - I tried running the tests again a few hours later and everything works flawlessly, all tests pass locally: both with the parallec master branch and with my PR branch.

jeffpeiyt commented 9 years ago

Hi Lukasz, Thanks much for your checking on this again. yes this may be tricky; maybe due to issues in the Travis environment. Travis is JDK1.7 I feel the sample Http server fails to run with the new pom.

Will try to reproduce it.

my local is MAX OS is os x yosemite. and currently run is fine.

The sshWorkerFakeVmPasswordTest may be easier to resolve: currently it is too strict on the expected error; probably sometimes the error output is slightly different. I would create a seperate issue, if you can rerun just this test, and then print the output, we will know the other error string.

https://github.com/eBay/parallec/blob/master/src/test/java/io/parallec/core/main/ssh/ParallelClientSshTest.java#L32

jeffpeiyt commented 9 years ago

Lukasz, if you can print out the output, we can fix this. https://github.com/eBay/parallec/issues/10 was created to track the issue regarding the sshWorkerFakeVmPasswordTest.

jeffpeiyt commented 9 years ago

Now I see the root cause. In the travis (maybe due to docker) the server starts too late (too slow) to accept the request. (maybe with the changes some library late init) . let me update the unit tests first