test execution time is not accurate

[bearxc]

Hi, I am running java/maven/JUnit4 on multiple test classes from testparameterinjector 1.14 , I do find the first method of first test class execution is very slow , it takes more than 3-4 mins compared with Junit Parameterized , any suggestion on this ?

I can not find this delay on sequently method/class .

[nymanjens]

Can you replicate this issue in a minimal example and share that example, please?

[bearxc]

the test project is running against our internal spring context /bean app , it is very hard to have a simple example.

we found 3-4 mins increase for first test method in Junit test xml , does testparameterinjector overwrite the way calculating test execution time ?

[nymanjens]

does testparameterinjector overwrite the way calculating test execution time ?

Not that I know

[bearxc]

we add time stamp on before/after method , execution time is not different , but execution time from junit report has increased a lot .

[bearxc]

I take a quick look on PluggableTestRunner ,
getTestMethodProcessors().postProcessTestInstance(testInstance, testInfo);

I think this method would take a longer time to process at the first time , considering there are multiple inherence in test class , and each test class have spring beans such as @Autowired private TestA testa; ...... @Autowired private TestB testb;

It would be slow at beginning when going through on parent classes from java reflection .

[nymanjens]

What would you suggest to fix this?

[bearxc]

Maybe we could exclude parsing /scan effort as part of test case execution itself , that cause confusion .

Also , it looks like PluggableTestRunner is running postProcessTestInstance/scan on every test method which is ver heavy.

Not sure how JUnitParamsRunner handle this .

[bearxc]

One more suggestion is maybe we need move parse/scan out of methodBlock , otherwise Junit treats that as part of method execution time.

Ideally , we need parse/scan/inject before test class start .

[nymanjens]

So if I understand you correctly, this is more about the confusing timings reported by JUnit than it is about the actual slowness?

I do wonder why it is so slow for you though. Do you have many test methods? I'd think that the @Autowired stuff gets done after the TestParameterInjector stuff. It would really be helpful if you have a minimal test that you can share that reproduces this issue, so that I can investigate where all the time is being spent. There is definitely a lot of work that's happening multiple times that could be cached if the need arises.

About the time reporting: I had a look at JUnitParamsRunner, and they seem to be interfacing with JUnit in a completely different way, and it's a bit hard to see which one is best.

If you have a proof of concept of what we could do differently, I'm happy to run it against all Google tests and see if it works.

[bearxc]

yes, you are right , sorry for confusion , that is not slowness actually , I changed title just now .

Our test classes are inherence from many layers, each layer is spring context with huge @Autowired , there are around 10000 test class/context need to be initialized, it takes 3-4 mins , we compared between JUnitParamsRunner and testparameterinjector , that initialization time is same , only difference is , JUnitParamsRunner trigger initialization before test class start while for testparameterinjector , it happens after test class start .

I think JUnitParamsRunner handle different way as yours for sure .

We will take a look and ping you back once we have good solution .

Thanks for help and support !!!