search

eBay / parallec

Fast Parallel Async HTTP/SSH/TCP/UDP/Ping Client Java Library. Aggregate 100,000 APIs & send anywhere in 20 lines of code. Ping/HTTP Calls 8000 servers in 12 seconds. (Akka) www.parallec.io
Apache License 2.0
805 stars 173 forks source link

plugins for clients #23

Closed tecbea closed 8 years ago

tecbea commented 8 years ago

Hi great work.

how to make plugins to create our own tasks?

a task can be a persistent connection ?

best

jeffpeiyt commented 8 years ago

@tecbea

Thanks for your interests in Parallec

how to make plugins to create our own tasks?

Currently Parallec supports HTTP/TCP/SSH/Ping as the tasks; adding a generic task is possible; however the interface from the builder needs to be defined to pass into the task needed input parameters. This needs some work.

May I ask what is your specific needs? For HTTP you may overwrite and construct and use your own AsynHttpClient http://www.parallec.io/docs/submit-task/#apis-on-http

setAsyncHttpClient() Optional Embed fast one from the store You may overwrite the client to your customized one for each task. The default one is the embed fast one from HttpClientStore.

a task can be a persistent connection ?

Yes. I believe the HTTP 1.1 is by default keep-alive.

Adding the response headers printing to the http worker and run this test: #24

https://github.com/eBay/parallec/blob/master/src/test/java/io/parallec/core/main/http/request/template/ParallelClientVarReplacementHostSpecificTest.java#L135

            //TODO
            System.err.println("PRINT headers:");
            logger.info("response headers:");
            for (Map.Entry<String, List<String>> entry : headers.entrySet()) {
                logger.info(" - {} = {}", entry.getKey(), entry.getValue());
            }
            logger.info("content type '%s'", response.getContentType());

We can find it is by default using keep alive. And you can check more options with Async Http Client by loading your own Async Http Client or change request headers to customize.

Please Let me know if this answers your question. We are very glad to assist with your needs.

Thanks

09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker - response headers:
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Cache-Hits = [0]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Cache-Hits = [0]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Age = [0]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Age = [0]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Fastly-Request-ID = [f96e9d28a95e6f1bb08e4c6af54b20549c7bc37d]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Fastly-Request-ID = [4042d3221ef328d988e73b63b9d49c4aad36b0c1]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Content-Length = [2]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Content-Length = [2]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Expires = [Wed, 16 Dec 2015 17:18:30 GMT]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Expires = [Wed, 16 Dec 2015 17:18:30 GMT]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Served-By = [cache-lax1421-LAX]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Served-By = [cache-lax1430-LAX]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Last-Modified = [Wed, 02 Dec 2015 19:12:23 GMT]
09:08:30.292 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Last-Modified = [Wed, 02 Dec 2015 19:12:23 GMT]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Connection = [keep-alive]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Connection = [keep-alive]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Server = [GitHub.com]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Server = [GitHub.com]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Cache = [MISS]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Cache = [MISS]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-GitHub-Request-Id = [C71B4F21:448E:110CD362:56719A8D]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-GitHub-Request-Id = [C71B4F1C:448E:110CD372:56719A8D]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Cache-Control = [max-age=600]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Cache-Control = [max-age=600]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Date = [Wed, 16 Dec 2015 17:08:30 GMT]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Date = [Wed, 16 Dec 2015 17:08:30 GMT]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Vary = [Accept-Encoding]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Vary = [Accept-Encoding]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Access-Control-Allow-Origin = [*]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Access-Control-Allow-Origin = [*]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Via = [1.1 varnish]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Via = [1.1 varnish]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Accept-Ranges = [bytes]
PRINT HEaders!!
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Accept-Ranges = [bytes]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Content-Type = [text/plain; charset=utf-8]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - Content-Type = [text/plain; charset=utf-8]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Timer = [S1450285710.222828,VS0,VE80]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker -  - X-Timer = [S1450285710.219623,VS0,VE80]
09:08:30.293 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker - content type '%s'
09:08:30.294 [AsyncHttpClient-Callback] INFO  i.p.c.a.HttpWorker - content type '%s'
09:08:30.298 [ParallecActorSystem-akka.actor.default-dispatcher-6] INFO  i.p.c.a.ExecutionManager - 
[1]__RESP_RECV_IN_MGR 1 (+1) / 2 (50.000%)  AFT 0.243 S @ API_1 @ 2015.12.16.09.08.30.296-0800 , TaskID : 8f3255c7-756 , CODE: 200 OK, RESP_BRIEF: 63 
09:08:30.298 [ParallecActorSystem-akka.actor.default-dispatcher-6] INFO  i.p.c.TestBase - 
!!Temperature: 63 TargetHost: API_1
09:08:30.299 [ParallecActorSystem-akka.actor.default-dispatcher-6] INFO  i.p.c.a.ExecutionManager - 
[2]__RESP_RECV_IN_MGR 2 (+0) / 2 (100.00%)  AFT 0.246 S @ API_0 @ 2015.12.16.09.08.30.299-0800 , TaskID : 8f3255c7-756 , CODE: 200 OK, RESP_BRIEF: 74 
09:08:30.299 [ParallecActorSystem-akka.actor.default-dispatcher-6] INFO  i.p.c.TestBase - 
!!Temperature: 74 TargetHost: API_0
jeffpeiyt commented 8 years ago

BTW, looks like AHC's implementation is using ConnectionsPool to reuse connection https://asynchttpclient.github.io/async-http-client/apidocs/com/ning/http/client/ConnectionsPool.html

jeffpeiyt commented 8 years ago

Hope this helps. Looks like there is no further discussion on this. Close it now. Feel free to reopen it if necessary.