Losing class test coverage report from under process-test-coverage

[paulbors]

I noticed that if I run a single class I get the expected coverage report just fine, odly enough is always the last test class that run. This happens on both Windows and Linux via IntelliJ and Maven.

activity.cfg.xml in test resources set to:

<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">

    <bean id="processEngineConfiguration"
          class="org.camunda.bpm.extension.process_test_coverage.junit.rules.ProcessCoverageInMemProcessEngineConfiguration">
        <property name="jdbcUrl" value="jdbc:h2:mem:activiti" />
        <property name="databaseSchemaUpdate" value="true" />
        <property name="jobExecutorActivate" value="false" />
        <property name="mailServerPort" value="5025" />
        <property name="enableExpressionsInAdhocQueries" value="true" />
    </bean>

</beans>

Test class signature:

@Deployment(
        resources = {"myProd.bpmn"}
)
public class ***Test extends ***BpmnBaseTest { }

public class ***BpmnBaseTest extends WorkflowsBaseTest {
    // Engine
    protected RuntimeService runtimeService;
    protected ProcessEngine processEngine;
    protected TaskService taskService;
    protected TaskQuery taskQuery;
    // Test Case
    protected ProcessInstance processInstance;
    protected ManagementService managementService;
    protected JobQuery jobQuery;
    // Test Data
    protected String businessKey;

    @Before
    public void setup() {
        runtimeService = processEngineRule.getRuntimeService();
        processEngine = processEngineRule.getProcessEngine();
        businessKey = getClass().getSimpleName() + "_" + new DateTime();

        // Always invoke the default start event
        processInstance = runtimeService.startProcessInstanceByKey(
                "my-workflow-id", businessKey
        );
        assertThat(processInstance).isStarted().hasProcessDefinitionKey("my-bpmn-workflow-id");

        taskService = processEngineRule.getTaskService();
        taskQuery = taskService.createTaskQuery();

        managementService = processEngine.getManagementService();
        jobQuery = managementService.createJobQuery().processInstanceId(processInstance.getId());
    }
}

public class WorkflowsBaseTest {
    @ClassRule
    public static ProcessEngineRule processEngineRule = TestCoverageProcessEngineRuleBuilder.create().build();
}

Camunda v7.10 community jUnit v4.12 SpringBoot v2.1.3.RELEASE Java v8 camunda-bpm-process-test-coverage v0.3.2

Please advice!

[paulbors]

My hunch is that the target/process-test-coverage folder is being recreated each time a test class is executed.

[paulbors]

If if run one test class at a time, I do see the class folder being added to the target/process-test-coverage folder one by one. When I run them all at once... only the report folder from the last class that run exists.

[ingorichtsmeier]

Hi @paulbors,

how do you execute your tests? From your IDE or with maven? Are all tests successfully?

Running mvn clean test should give you the desired result.

[paulbors]

All test pass just fine (I just wrote them). It happens on both the IDE (IntelliJ IDEA Ultimate 2018.1) and Maven on localhost Windows 10 as well as Linux via Jenkins by cleaning up the working space and performing a fresh checkout.

On localhost via the IDE, I had to run one test class at a time in order to get the test coverage folder added one at a time per test class. If I run All the tests via maven or in my IJ as JUnit run config... Only the output of the last folder is present.

Where in the code do you guys create/write the test class coverage report? I would like to glimpse over that logic.

[paulbors]

Keep in mind my test config shuts down the workflow engine and starts a new one per each test class I run.

[ingorichtsmeier]

Hi @paulbors ,

Maybe the root cause is in your project setup/Class hirachy?

I've tried it with this example: https://github.com/camunda-consulting/simple-junit-test. Two processes, two test classes, all coverage is collected in the target/process-test-coverage folder after running mvn clean test.

[zambrovski]

It seems to be a problem if the rule is declared in the super class... See #43

[zambrovski]

Hi @paulbors

I examined your use case. Here is what I found out.

There are two modes how you can use the rule:

• use it as a simple @Rule so it gets initialized and generates method coverage only.
• use it as @ClassRule and @Rule and you will get class coverage along with method coverage.

In the second case, the rule is static and is initialized once (actually on the first run) which means it will get the class name from the first inherited class and will generate one class report for this. In the same time, it will collect all methods from all classes inheriting from the same class in that report. So I could reproduce your issue. This is not really a bug, but more the issue of static rule. My advice is not to use static class rule to a super class if you want to get report per class - it is by the way just fine to do so if you want to separate your tests in classes but interested in one aggregated report.

I'll close the issue for now.