How to send multiple requests (say, 1 million) in parallel to the same target?

[Nesh108]

Hello,

I would like to use Parallec to test my servers. I would like to be able to query a certain amount of endpoints of my website and get some logs out, here is what I currently have:

        ParallelClient pc = new ParallelClient();

        Map<String, Object> responseContext = new HashMap<String, Object>();
        final ArrayList<ResponseData> responses = new ArrayList<ResponseData>();

        pc.prepareHttpGet(targetEndpoint)
                .setHttpPort(targetPort)
                .setAutoSaveLogToLocal(true)
                .async()
                .setConcurrency(999999999)
                .setReplaceVarMapToSingleTargetSingleVar(varName, listCities,
                        targetURL).setResponseContext(responseContext)
                .execute(new ParallecResponseHandler() {
                    public void onCompleted(ResponseOnSingleTask res,
                            Map<String, Object> responseContext) {
                        responses.add(new ResponseData(res));
                    }
                });

I am unsure of the meaning of .setConcurrency(int). Is the higher the better? I just want to have the highest speed possible.

Also, does .async() allow me to send all the request in parallel and waiting for the responses afterwards?

Lastly, how can I tell parallec to send X times those requests?

Thanks in advance!

[Nesh108]

Addition: I tried using this with a while loop and I calculated that sending 1,000,000 request would take around 28 hours, which definitely too much!

I found Apache JMeter to have good performance but I would like to have more flexibility and being able to code stuff around it, so I would prefer using Parallec for this.

[jeffpeiyt]

Hi Nesh,

Sorry I missed this from the git notice earlier. Thank you for trying out Parallec! Please let me know if you have more questions.

Q: I am unsure of the meaning of .setConcurrency(int). Is the higher the better? I just want to have the highest speed possible.

A: the concurrency is about the sliding window size to control how fast to send the requests. The default is 1000, which gives pretty good performance. With limited bandwidth and file descriptors we may not set it that big: config Check here for actor model. Note that configs and timeouts may impact the performance significantly. check more on config/timeout

In our test we complete 8K http response (to distinct target hosts) in 12 seconds. So I million hosts would need 25 minutes. 1000000/(8000/12)/60 = 25 min.

Q: Also, does .async() allow me to send all the request in parallel and waiting for the responses afterwards?

A: Async() allow you to run in a none blocking way and then poll progress. check here for example. In the beginning you may run sync model for simpler code execution.

[jeffpeiyt]

For multiple requests to the same target: try following this example

[jeffpeiyt]

closed for now as there is no further activity. For more questions feel free to reopen.