Switch to use relative paths in OpenCover report #1315

Closed RedlineTriad closed 5 months ago

I am running coverlet in a self-hosted GitLab instance to collect code coverage for Sonarqube. Problem is, we have Windows runners and the test job and the sonar job can run in different absolute paths. This causes Sonarqube to be unable to calculate code coverage because the absolute paths from the first job point nowhere.

This is the command I am using:

coverlet $TEST_PROJECT\bin\Debug\$TEST_PROJECT.dll \
--output "./TestResults/" \
--target nunit3-console \
--targetargs "$TEST_PROJECT\bin\Debug\$TEST_PROJECT.dll --result ./TestResults/TestResult.xml" \
--format opencover

This is what the report looks like (with anonymized paths):

<File uid="1" fullPath="C:\Gitlab-Runner-02\builds\Fu_ads7S\0\APP_PROJECT\File1.cs"/>
<File uid="2" fullPath="C:\Gitlab-Runner-02\builds\Fu_ads7S\0\APP_PROJECT\File2.cs"/>
<File uid="3" fullPath="C:\Gitlab-Runner-02\builds\Fu_ads7S\0\APP_PROJECT\File2.cs"/>

How can I make coverlet generate the opencover with paths relative to execution/project instead of absolute ones? I looked at multiple issues similar to this one but didn't see any resolution on what to do. I also read the docs, but the only options I saw --use-source-link or somehow using deterministic builds. With source link enabled it still generated absolute paths and I haven't tried deterministic build because it didn't seem straightforward.

I am having a similar problem, but only on one project.

My other projects have the base path in source and relative paths in filename.

I can't find any difference in setup/config. It's driving me mad!!!

Here's an old issue we can hopefully refer back to, to see where this is handled and maybe get some ideas what may be going wrong: https://github.com/coverlet-coverage/coverlet/pull/661/files

I'm on my mobile at the moment so can't check.

💥 sussed it... reading through the code I noticed:

var splittedPaths = group.Select(absolutePath => absolutePath.Split(Path.DirectorySeparatorChar))
    .OrderBy(absolutePath => absolutePath.Length).ToList();

if (splittedPaths.Count == 1)
{
    return group.Key;
}

And my problematic project has only 2 classes, one of which is tagged with [ExcludeFromCodeCoverage]- just leaving on file/class.

I removed the data annotation and voila- the cobertura report has a valid source and uses relative paths.

Should we consider this a bug or find a way to tackle it? 🤔

Not sure what you mean? I don't have ExcludeFromCodeCoverage as an annotation anywhere.

This was just an explanation of my problem/solution.

I'm sure yours wont be exactly the same but hopefully you can use some of the above information to help you diagnose/troubleshoot.

Looking at your example, it shows at least 3 files, so I don't think your issue relates to only having one class being analysed.

Are you able to share a link to the GitLab project? (I suspect not- but just in case).

Can you share your command-line? And perhaps more/all of the generated cobertura report?

Are you able to share a link to the GitLab project? (I suspect not- but just in case).

Sadly no.

Can you share your command-line? And perhaps more/all of the generated cobertura report?

Command line is in the issue, I can see about entire report, will have to anonymize it a bit.

Ah yes, forgive me. So this is actually opencover and my problem was with cobertura.

I can't see anything in https://github.com/coverlet-coverage/coverlet/blob/master/src/coverlet.core/Reporters/OpenCoverReporter.cs to attempt to rationalize, so I think this would need adding (could copy the code from cobertura, although we'd likely want to try and DRY it up while we're at it)

Do you know if you could use cobertura format instead to get around your issue?

I don't really know anything about Sonarqube.

Do you get test coverage in the GitLab UI? That's what we get by generating the cobertura report and collecting as the approperiate aritficat:

Sadly Sonar doesn't have support for Cobertura for C# code coverage: https://docs.sonarqube.org/latest/analysis/coverage/

So I would have to just implement the normalization for OpenCover to fix it?

@RedlineTriad I don't know all the features for Sonarqube, do you know if they support deterministic report https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/DeterministicBuild.md#deterministic-report?

At the moment we support it only for cobertura report, but we're planning to support it also for others.

This is the tracking PR https://github.com/coverlet-coverage/coverlet/issues/1075

@RedlineTriad Maybe you could open an issue on SonarQube side too because it seems (from the link you pasted) that they do support covertura for both Flex and Python so I guess it should not be too much of a trouble to add it for C#.

Thanks @Evangelink @RedlineTriad , I've mentioned this to our PM (about adding support for the cobertura format for the Sonar C# offering).

Same problem here. My workaround (for linux): Runner1:

run tests and generate report to file opencover.xml
sed -i "s|$PWD|CURRENTDIRECTORYREPLACEMENT|g" opencover.xml

Runner2:

sed -i "s|CURRENTDIRECTORYREPLACEMENT|$PWD|g" opencover.xml
run sonar scanner

Explanation: $PWD - variable with current path sed -i "s|$PWD|CURRENTDIRECTORYREPLACEMENT|g" opencover.xml - replace all occurrences of current path with string "CURRENTDIRECTORYREPLACEMENT" sed -i "s|CURRENTDIRECTORYREPLACEMENT|$PWD|g" opencover.xml - replace all occurances of string "CURRENTDIRECTORYREPLACEMENT" with current path After that when i run sonar scanner i got opencover.xml with correct paths

This issue is stale because it has been open for 3 months with no activity.

This issue was closed because it has been inactive for 9 months since being marked as stale.

coverlet-coverage / coverlet

Switch to use relative paths in OpenCover report #1315