search

microsoft / vstest

Visual Studio Test Platform is the runner and engine that powers test explorer and vstest.console.
MIT License
894 stars 321 forks source link

dotnet test --collect:"code coverage" includes 3rd party dlls #1917

Closed GoFightNguyen closed 5 years ago

GoFightNguyen commented 5 years ago

Description

.NET Core 2.1 MSTest.TestAdapter v1.4.0 MSTest.TestFramework v.1.4.0 Microsoft.NET.Test.Sdk v 15.9.0

Steps to reproduce

Create a .NET Core Test project with a test Add a dependency to the FluentAssertions NuGet pkg dotnet test --blame --collect:"Code Coverage" Open the coverage file

Expected behavior

I would not see coverage details for FluentAssertions

Actual behavior

I see coverage details for FluentAssertions

GoFightNguyen commented 5 years ago

I discovered the .runsettings file. At the solution level, I created a codecoverage.runsettings file. I then run this command: dotnet test --blame --collect:"Code Coverage" --settings:..\codecoverage.runsettings

This appears to give me what I need. Here are my runsettings. Hopefully I'm not leaving out anything important.

`<?xml version="1.0" encoding="utf-8"?>

.*\mydomains.*.dll .*tests.dll `
nigurr commented 5 years ago

@GoFightNguyen Currently we do instrument all dependent dlls (whether they come from 3rd part dependency or nuget reference). In this case we try to collect code coverage for FluentAssertions.

However this package can be excluded from coverage report by modifying the runsettings as you have mentioned. This is working as designed :)

gioce90 commented 2 years ago

Exclusion by .runsettings file is a bit painful. Exclude 3rd party and Tests assemblies it would be nice.

BradleyBarnett commented 1 year ago

Agree 99% of the time no one would want to include nuget packages or test assesmblies, this should be default behaviour.

cliffAtOsaic commented 4 months ago

I agree 100% the default behavior should not include extra DLLs. Futhermore, filtering by .runsettings file is not working for both "dotnet test" and "dotnet-coverage". This should be reopened and the default behavior should be changed. Alternatively, an easy switch should be added to remove 3rd party imported NuGets/DLLs etc.

LoupaLoupa commented 1 day ago

This should be reopened as a feature request, I can't fathom reasons the default behavior should be to include assemblies we have 0 control over

cliffAtOsaic commented 1 day ago

@GoFightNguyen Currently we do instrument all dependent dlls (whether they come from 3rd part dependency or nuget reference). In this case we try to collect code coverage for FluentAssertions.

However this package can be excluded from coverage report by modifying the runsettings as you have mentioned. This is working as designed :)

This is not working as designed, because the runsettings does not exclude extra DLLs even when explicitly told to do so.

(Nor does this make any sense to have this as the default behavior.)