quick-perf / quickperf

QuickPerf is a testing library for Java to quickly evaluate and improve some performance-related properties
https://github.com/quick-perf/doc/wiki/QuickPerf
Apache License 2.0
488 stars 65 forks source link

Support JUnit5 test template #114

Closed loicmathieu closed 4 years ago

loicmathieu commented 4 years ago

Support JUnit 5 test template: https://junit.org/junit5/docs/current/user-guide/#writing-tests-test-templates

Tests has been integrated with @RepeatedTest that is a build-in test template extension for repeated test. Parameterized test should also works.

As for dynamic test, the annotation will apply for each test invocation derived from the test template method.

jeanbisutti commented 4 years ago

Thank you for this PR @loicmathieu

Just a few things:

1) Could you please rebase on master

2) To let TestExecutionContext almost immutable, I would rather to not add a setRunnerAllocationOffset(int runnerAllocationOffset) method in TestExecutionContext. Something like this could be used (the added tests are always green):

@Override
    public void beforeEach(ExtensionContext extensionContext) {

        boolean testTemplate = isTestTemplateUsed(extensionContext);
        int junit5AllocationOffset = testTemplate ? 0 : 40;

        testExecutionContext = TestExecutionContext.buildFrom(quickPerfConfigs
                                                            , extensionContext.getRequiredTestMethod()
                                                            , junit5AllocationOffset);
    }

    private boolean isTestTemplateUsed(ExtensionContext extensionContext) {
        Method testMethod = extensionContext.getRequiredTestMethod();
        Annotation[] methodAnnotations = testMethod.getAnnotations();
        for (Annotation methodAnnotation : methodAnnotations) {
            if(testTemplateFor(methodAnnotation)) {
                return true;
            }
        }
        return false;
    }

    private boolean testTemplateFor(Annotation methodAnnotation) {
        return methodAnnotation.annotationType().equals(RepeatedTest.class);
    }

3) Please rename AllocationAnnotationsJUnit5Test class in something more specific, for example AllocationJUnit5TestTemplateTest

loicmathieu commented 4 years ago

@jeanbisutti I rebase on master and rename the test. Unfortunatly I keep the allocation offsert management as is as a test template is a method annotation with @TestTemplate or any annotation that extends it, and there is no easy way to know by reflection that an anotation extends another (surprinsingly, TestTemplate.class.isAssignableFrom(RepeatedTest.class) returns false, this should have been the canonical way to check that an annotation extends another).

If you think of another way to do it I'm open for it, but checking all annotations and all parent of these annotations recursively seems to be a bit overkill.

jeanbisutti commented 4 years ago

Thank you. Ok for merging the PR like this.

For information, the allocation offset should not be stored in the test execution context, but the implementation is simpler.