search

coverlet-coverage / coverlet

Cross platform code coverage for .NET
MIT License
2.96k stars 386 forks source link

CD pipeline, running against dlls and coverage generation on a different machine to the build machine #1364

Open danpowell88 opened 2 years ago

danpowell88 commented 2 years ago

I'm trying to setup a CI/CD pipeline where we reduce the number of builds we do and just run our unit tests agains the built source, potentially on a different machine to the machine that built the dll itself (and almost definitely in a different checkout directory).

So

Machine #1 (working directory c:\build{GUID})

dotnet restore
dotnet build MySolution.sln
dotnet publish UnitTests/UnitTest.csproj -o output/tests/unittests

/output/tests/unittests is then published as a build artifact

I've also included the dotnet reproducible builds package (and building using TeamCity)

Machine 2 (working directory c:\build{GUID2}) Copy build artifacts to /output/tests/Api.UnitTests

dotnet test output/tests/Api.UnitTests/UnitTests.dll --logger:TeamCity --collect:"XPlat Code Coverage" --ResultsDirectory:output/results/tests -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura`

The tests run correctly however no coverage is output for them. I did notice that the CoverletSourceRootsMapping is not copied into the output when I do a dotnet publish, I have added a step to manually copy this into the publish directory and I can see the coverage then works, but only if the test coverage is run on the same machine that the build ran on.

What am I missing here, should I give up and just revert back to building everything and running the coverage all together ?

petli commented 2 years ago

Hi @danpowell88, coverlet relies on finding the source code files to determine what modules to instrument. Without the source code in the exact same path on the machine running the tests coverlet will consider all the modules to be third-party packages and skip them. #1164 discuss a parameter to give the user control over this, and #1360 implements a variation of that idea to workaround this issue.

So right now you would have to build and execute the tests in one go, but keep an eye on those issues for a fix.

MarcoRossignoli commented 2 years ago

@petli is right, you could dogfood the nightly build and test that feature for us also https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/ConsumeNightlyBuild.md