search

jenkinsci / peass-ci-plugin

Jenkins plugin for peass to support performance measurement in CI
https://plugins.jenkins.io/peass-ci/
GNU Affero General Public License v3.0
4 stars 12 forks source link

Better display of unexecuted tests (because of generateCoverageSelection: true) in Peass Overview #199

Closed mawHBT closed 2 years ago

mawHBT commented 2 years ago

What feature do you want to see added?

If you set generateCoverageSelection: true and e.g. two TestCases call changes in a commit but only one TestCase is executed for measurement (because it calls the changes more often than the other), you see an error in Peass Overview Results for the unexecuted TestCase. This is not really an error. The TestCase was not executed for a reason. Field telling you "unmeasured" should maybe be yellow instead of red. So you will recognize that the TestCase was not executed because measurements took place using the TestCase which calls the changes more often.

Upstream changes

No response

DaGeRe commented 2 years ago

This should work since https://github.com/jenkinsci/peass-ci-plugin/commit/3fa7455bffe4c2d7ddafd17a81820b341365e626, can you confirm this?

mawHBT commented 2 years ago

If a test is not executed because another one calls the changes more often, in overview you now read: "This test was not selected by dynamic selection: COVERAGE_SELECTION" So this works.