search

abstracta / jmeter-java-dsl

Simple JMeter performance tests API
https://abstracta.github.io/jmeter-java-dsl/
Apache License 2.0
462 stars 58 forks source link

JMeter 5.6 upgrade #235

Open sergey-morenets opened 9 months ago

sergey-morenets commented 9 months ago

Hi

It seems that jmeter-java-dsl uses Apache JMeter 5.5 (as seen in pom.xml):

<jmeter.version>5.5</jmeter.version>

But JMeter 5.6 was released 4 months ago. Do you have any plans/ETA to migrate to the current (5.6.2) version? It will allow to resolve long-awaiting Gradle issue:

https://github.com/apache/jmeter/issues/5322

rabelenda commented 9 months ago

Hello Sergey, thank you for bringing this!

Yes, 5.6 had some issues after it was released, and then 5.6.1 had some others, and now 5.6.2 has one that seems to be related to darklaf. So we were waiting mainly for all main issues to be fixed before considering moving to 5.6.x.

The one thing that has me wondering is the change in 5.6.1 about default http encoding, which sounds to me to be a non backwards compatible change (test plans relying in OS default encoding might stop working when the default encoding is changed to UTF-8). We would need to try the latest version and run some tests, and if the change is actually non backwards compatible, I guess we would need to release it as 2.0 for proper semantic versioning, even though we have planned several non backwards compatible changes for 2.0 (which would require some time yet to implement), we probably would need to move them to 3.0.

I didn't know about the particular fix for gradle. There are also several updated dependencies and performance fixes that are very interesting and important.

We will try newest version and try to make a release soon including 5.6.x release.

Thanks!

sergey-morenets commented 9 months ago

Hi @rabelenda

Thank you for the explanation. Can you please update the documentation (https://abstracta.github.io/jmeter-java-dsl/guide/#setup)? It contains very huge piece of configuration for Gradle. And I tested more compact solution that worked with Gradle 8.4:

    testImplementation("us.abstracta.jmeter:jmeter-java-dsl:1.23") {
        exclude("org.apache.jmeter", "bom")
    }
rabelenda commented 8 months ago

Sure, thank you!

FakeLogin commented 4 months ago

Just checking in - are the bug fixes we were waiting for included in JMeter 5.6.3? Another reason I'm curious about JMeter 5.6.3 is that it potentially addresses some dependency vulnerabilities through its updates.

rabelenda commented 1 month ago

Hello, thank you for asking about this.

This is something that is taking more time than initially expected.

We have discussed about this in discord as well, but we still see some breaking changes introduced in JMeter 5.6, which is an exceptional case since we haven't seen such type of changes in the past with JMeter, with plugins we use in JMeter DSL (like Parallel Controller) which would require additional changes other than updating the version.

The other change that is significant, from our point of view, is the default encoding changed to UTF-8. We consider this change to be non backwards compatible, and are talking with JMeter team, but still need to agree on this with JMeter team and submit a PR to JMeter to revert it, do not consider it a non backwards compatible change, or release a new major version.

We have several courses of action that we need to evaluate and invest more time to define the final option.

If we consider the change in encoding to be non backwards compatible (still need to invest a little more to double check this), then we would need to release a 2.0 version to properly reflect this nature to JMeter DSL users. Among the required changes we would need to fix all plugins and jmeter dsl logic to adapt to the changes done in JMeter 5.6.

If we release 2.0, then we also would be inclined to include some additional non breaking changes that we have planned to integrate in version 2.0. If we include those, we need to provide users with some manual or automated mechanism for migrating to the new api. These changes also introduce additional investment and effort we need to spend. We could decide to still hold such changes for 3.0, but 2.0 would be a good chance to introduce them.

Finally, JMeter team has started talking about migrating to Java 11 and plan for JMeter 6. I don't know of any ETA for that. If such release is not long ahead in time, we could take advantage of that and just migrate everything to JMeter 6 and use latest Java 11 features. This would also require changes in plugins and changes already included in 5.6 on top of any changes that may be introduced in JMeter 6. We could also introduce all necessary changes we have been holding for 2.0, or plan for versions 3.0 and/or 4.0.

There is a balance between fixing things, or keeping up to date, fast and avoiding users to have several non backwards compatible changes in a short period of time. We need to invest more time to figure the correct plan of action.

Does anyone have any ideas or insights that can help us in such decision? Anyone willing to help in start implementing such changes?