Requests are executed in separate streams but sequentially

[serputko]

Hello all, right now part of our project migrates from http1.1 to http2 and we need to generate server-side load to the server. I've started using http2 jmeter plugin but I doubt if it works as expected according to http2 spec. My expectation is to interact with server like browser does: send packets concurrently using multiplexing and streams via single tcp connection. Right now I observe next behavior: ++ I'm able to send requests via single tcp connection, i see that different samplers have different streams ids -- Requests are processed sequentially. See pictures for network activity from proxy and also threads state during test execution (Blue is network activity, Yellow- wait)

Am i missing something or that is expected design and it should work in this way?

Thanks

[serputko]

Also wanted to mention that parameters of SETTINGS frame are different for chrome and jmeter, is there any properties to manage them?

Browser(chrome) SETTINGS_HEADER_TABLE_SIZE = 65536 SETTINGS_ENABLE_PUSH = 0 SETTINGS_MAX_CONCURRENT_STREAMS = 1000 SETTINGS_INITIAL_WINDOW_SIZE = 6291456 SETTINGS_MAX_FRAME_SIZE = 16384 SETTINGS_MAX_HEADER_LIST_SIZE = 262144 Jmeter SETTINGS_HEADER_TABLE_SIZE = 4096 SETTINGS_ENABLE_PUSH = 1 SETTINGS_MAX_CONCURRENT_STREAMS = 32 SETTINGS_INITIAL_WINDOW_SIZE = 8388608 SETTINGS_MAX_FRAME_SIZE = 16384 SETTINGS_MAX_HEADER_LIST_SIZE = unlimited Server SETTINGS_HEADER_TABLE_SIZE = 4096 SETTINGS_ENABLE_PUSH = 1 SETTINGS_MAX_CONCURRENT_STREAMS = 128 SETTINGS_INITIAL_WINDOW_SIZE = 65536 SETTINGS_MAX_FRAME_SIZE = 16777215 SETTINGS_MAX_HEADER_LIST_SIZE = unlimited

Thanks

[3dgiordano]

Hi @serputko

Related to the SETTINGS frame, please, create a separate issue. With this we can give you a proper follow-up.

Related to the sequential of the request model. You are correct, the execution model actually follow the common sync model and it is a limitation for the simulation you want to perform..

This is a regression, in our previous version 1.6.2 asynchronous execution was supported. We will be working to provide similar support in the new version 2.0.

Thank you very much for reporting the problem to us.

Regards

[serputko]

Basing on our conversation let me summarize here some key points.

Must have features for http2 client: 1) Possibility to "Retrieve all embedded resources" consecutively using http2 streams(managed by SETTINGS http2 frame) via 1 tcp connection rather than using 6 http connections like HTTP Request samples does. [Parallel downloads Numbe] value should not be used for http2 requests as concurrency is managed by SETTINGS frame BUT [Parallel downloads Number] could be used if http1 request is sent by HTTP2 Request sampler

2) Possibility to control execution of HTTP2 Request samplers in concurrent OR sequential way.

Why only "Retrieve all embedded resources" concurrently is not enough for real-life cases? Actually web application logic is not always sequential only or concurrent only, usually some requests are done one by one and another are processed in parallel. Here is the sample jmx of opening page action:

Sampler names describes what they does and how i expect them to be executed. And as you see from the timestamps in View Results Tree all http2 requests are processed one by one, even when i set httpJettyClient.minThreads=50. Right now there is no extra controllers except main transaction controller so I suppose plugin could provide some controllers and/or sampler properties to be used to control flow and scope of parallel execution and do we want to execute request sequentially or multiplexed(like it does in 1.6.2 version).

Possible scenario structure for the described case:

I've also found out that legacy plugin version 1.6.2 is able to send requests concurrently as well as make them sequential if required. That means that the scenario above could be implemented using legacy plugin version 1.6.2 BUT you will face all known functional and performance issues as well :)

Here is two implementations of the same page logic using plugins 1.6.2 and 2.0.2 1) Load html page with all static resources + 2 dynamic calls multiplexed(parallel) 2) load two resources sequentially after step 1 3) load 4 resources multiplexed(parallel) after step 2

Http2 plugin version 1.6.2 legacy Everything works as designed(except small delay after loading html page)

Http2 plugin version 2.0.2 everything is executed sequentially no way to implement expected realistic behavior

P.S. Why Parallel controller SHOULD NOT be used for parallel http2 requests execution: In jmeter to run request in parallel for single thread, the parallel controller could be used https://www.blazemeter.com/blog/parallel-controller-in-jmeter. BUT parallel controller is a mess in this case of using http2, it is creating new thread and new connection for each request in it's scope, at least it works like this for http1 regular sampler. So for 10 requests in Parallel controller - you'll get 10 new connections and 10 new threads. It also usually affects response times as some extra time is spent for connection setup while your real client won't do that. Imagine you have 5 requests with 40ms response each, total sequential time is 200ms You decide to put them in parallel like browser does and eventually you got total time 290 ms, why? because parallel controller created 5 new connections in parallel, spent 250ms for connection and 40ms for actual requests and as a result now you work even slower with parallel processing comparing to sequential :)

[3dgiordano]

Hi @serputko

We made an initial support implementation of asynchronous execution in the plugin. In this preview release, we add a new Controller named "bzm - HTTP2 Async Controller". This controller will allow you to run in parallel all the requests that you incorporate in there.

You can download and test the pre-release version in alpha in the releases section https://github.com/Blazemeter/jmeter-http2-plugin/releases/tag/v2.0.3-alpha

Let us know if it resolves what was reported in this issue.

[ck-singh]

Hi @3dgiordano, Thanks for getting the Async execution feature rolled out as part of v2.0.3-alpha.3 release of this plugin. I tried using the alpha and noticed that the response time reported by the parent transaction controller with multiple child "bzm - HTTP2 Async Controller" is incorrect as it seems to ignore the parallel execution behavior and add up response time of each individual sample. Is there any work around for capturing the correct response time details when using "bzm - HTTP2 Async Controller"?

[serputko]

@ceekay007 that's known one, try to use parallel controller with 1 thread and "generate parent sample" as a parent of Http2 async controller Like this:

You would get more valid response times

[ck-singh]

Thanks @serputko for your prompt response. I will give it a shot 👍

[3dgiordano]

Hi @ceekay007. Thanks for your feedback.

It's a problem in how the transaction controller performs the calculation. Since http2 does something unexpected, running requests concurrently, the transaction controller fails to calculate when summing. As @serputko mentions, there is a trick and that is to use the bzm - parallel controller which does it correctly. It is a workaround, the idea is to provide a controller in the future that allows doing what the transaction controller does wrong.

@ceekay007 I would appreciate if you report this issue in another new issue, so we don't contaminate the current thread. With that we can discuss how to handle the alternative and what the solution might be. We would also have a ticket associated with that functionality for when we can develop it.

[ck-singh]

Hi @3dgiordano, thanks for the additional context. I can definitely open a new issue, I refrained from doing it initially as I was not sure if the feature/fix is already being worked on as part of the wider http2 async controller GA release.

[ck-singh]

It seems @serputko has already opened a new issue.

[3dgiordano]

Hi @serputko

Final release of version 2.0.3 was done today. https://github.com/Blazemeter/jmeter-http2-plugin/releases/tag/v2.0.3

Soon it will be available in JMeter Plugins Manager.