The plugin does not send the mutation test report to SonarQube for maven multi-module project

[Alexander-Pavlenko]

I am using:

• pitest 1.4.9;
• SonarQube 7.9.2;
• mutation-analysis-plugin latest;
• multi-module maven project.

I see that pitest generates reports by default path (target/pit-reports). And if I use an absolute path to report-files it will work fine and the mutation-analysis-plugin will send the report about mutations. But it works only for one module. I have two modules with reports files. I have configured all the required properties for mutation-analysis-plugin. I have done next:

• defined path to parent module (absolute path);
• defined path to report file (relative path: target/pit-reports);
• enabled experimental feature;

But unfortunately, it does not work. And I did not get any results about mutations on SonarQube.

[gmuecke]

Hi Alexander, the plugin tries to determine the project root, which may fail (if you have some debug logs, that might give a clue). But you can also specify the root module of your multi-module project by using the property dc5.mutationAnalysis.project.root. Have you tried this?

i.e. mvn sonar:sonar -Ddc5.mutationAnalysis.project.root=/path/to/your/root

[Alexander-Pavlenko]

Hi Gerald, Actually, I have defined dc5.mutationAnalysis.project.root and according to logs, the plugin uses the correct path to root module. But it does not work. Also, I have a bit look through code and found some interesting in next strings: final Collection<Mutant> globalMutants = this.reportCollector.collectGlobalMutants(context); You found all mutans for all modules, but you don't use them for generating a report. Is it expected behavior?

[gmuecke]

Hi Alexander, The global mutants are only used for calculating some metrics. The reports are evaluated per module and aggregated on the server side for the project level statistics - which form the "global report". Some calculations require the global numbers (i.e. the TestKillRatioComputer), but the project-level statistics are actually a set of aggregated values from each submodule. As we're using the plugin for multi-module projects without any problems, my best bet would be a configuration issue. Are you using the aggregated reports feature of pitest and want to import this ( <goal>report-aggregate</goal>)? Or are you creating pitest reports per submodule?

[Alexander-Pavlenko]

Hi Gerald, Thank you for the clarification. We used to create pitest reports per submodule. But how I said it did not work. We configured dc5.mutationAnalysis.project.root and dc5.mutationAnalysis.pitest.sensor.reports.directorywith correct values. What other problems maybe with the configurations? Now we try to use the aggregated reports feature of pitest.

[gmuecke]

Hm, I don't know about other configuration issues, apart from a special maven proejct struture. We use it in a default project structure, with each submodule being a subfolder of it's parent, and it works without any special configuration magic.

Could you share your project or provide a minimal reproducer-project?

[Alexander-Pavlenko]

Hi Gerald, Thank you for help. But of course, I can not share my project or some project configuration, I will try to create some reproducer project but It will take a few weeks because we have a lot of specific configurations.

[StefanPenndorf]

Hi Gerald,

seems to me I have the same issue using

• SONAR 7.8 (build 26217)
• Pitest 1.5.2
• mutation-analysis-plugin 1.4
• multi-module maven project

I tried to display debug logs using maven -X flag in mvn sonar:sonar -X but the only line mentioning "mutation" is:

[DEBUG] 15:34:03.074   * Mutation Analysis 1.4 (mutationanalysis)

There is no single line mentioning "pitest" nor "dc5" in the log output.

(Side note to SONAR and mutation-analysis-plugin: SONAR dropped MySQL support in 7.9 thus we're tied to 7.8 at the moment, database change is planned but admin has no time, yet. Admin also reported that mutation-analysis-plugin version 1.5 could not be installed on SONAR 7.8 thus the "downgrade" to 1.4)

[gmuecke]

Hi Stefan, From the information you're providing its really hard to tell why it's behaving that way. Could you provide a re-reproducer? (a stripped-down project that shows the same behavior?)

[StefanPenndorf]

Hi Gerald,

sorry that I didn't get back earlier. I solved my problem in the meantime. It was a configuration problem on my side, I think: I didn't activate the mutator rules in our quality profile as described here: https://github.com/devcon5io/mutation-analysis-plugin/wiki/Installation#quality-profile

Even though I scanned the page after installing the plugin, I didn't recognize the ToDos in the first place. Additionally Jenkins Sonar Plugin was using an older version of Sonar Maven Plugin which didn't provide appropriate output. After upgrading I found warnings in the log stating I should active bug rules in Sonar. This is what helped me at the end:

1.) Using latest Sonar Maven Plugin by calling org.sonarsource.scanner.maven:sonar-maven-plugin:3.7.0.1746:sonar -Dsonar.host.url=**** -Dsonar.login=*** as a Jenkins Post-Build-Step (Call Maven Goals) 2.) (And I think this is the real cause) Activating the bug rules tagged "mutation-operator" (as described on the installation page)

[eriknellessen]

I also had the problem of not seeing results in SonarQube in a maven multi-module project. I used pitest-maven v1.5.2, mutation-analysis-plugin v1.5 and SonarQube v7.9.4.

I could find the cause of my problem. It was that I was doing the analysis on a short-lived branch. By creating a long-lasting branch, I could verify that the analysis was shown in SonarQube. I think that this will occur to other people. If you try a new tool, you usually do that on a feature branch. And then you will get the same problem as I did. Besides, analyses on feature branches would be pretty helpful. You could then evaluate the quality of the unit tests before merging into master.

@gmuecke Is there any plan to also do the analysis on feature branches?

[gmuecke]

@eriknellessen no, there aren't any plans.

The branch support is afaik a feature of the commercial editions of Sonarqube, and I don't have the time or money to implement this a feature for free to commercial users.

So you have two options:

• you implement it yourself and either keep it to you or contributing it to this project
• you consider paying me or someone else, to implement it

What do you think?

[eriknellessen]

@gmuecke I also only have the community edition of SonarQube available. You can create branches also in the community edition. I do so by adding the -Dsonar.branch.name=${env.BRANCH_NAME} command line parameter to the execution of mvn sonar:sonar.