Exclude file patterns and an assembly from CodeCoverage. Can FCC run in a BuildServer?

[aboimpinto]

Installed product versions

• Visual Studio: 2022
• This extension: 1.1.94

Description

I want to exclude from CodeCoverage all the files with the extension *.axaml.cs, and this is my settings.

In the end, the file with that extension is still included.

I want to exclude a whole assembly but need help finding a way.

I know I can use the [ExcludeFromCodeCoverage] tag, but I'm trying to use something other than that because that is too intrusive for the developer. I want to set this on the CodeCoverage settings that can be shared in all developer machines.

Can you also provide a way to export these settings to be consistent in every developer machine?

Is there a way to use this on the build server? It would be excellent for the output report to be generated during the build WorkFlow, and we can check it later.

Steps to recreate

Try to setup with the settings page.

Current behavior

I could not find a way to add an exclusion of a pattern or a whole assembly from CodeCoverage.

Expected behavior

I would expect the file patterns to be excluded, and a project could be excluded, and be able to generate a report from the Build Server

Thanks a lot Paulo Aboim Pinto

[aboimpinto]

From this discussion https://github.com/FortuneN/FineCodeCoverage/discussions/186 I found that I could exclude the project using the --exclude field, and now

I got the right project excluded.

[tonyhallett]

Note that if you switch to ms code coverage then you need to use that method of exclusion - https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#include-or-exclude-assemblies-and-members

Is there a way to use this on the build server? FCC only runs from within visual studio, initiated from the test explorer.

Can you also provide a way to export these settings to be consistent in every developer machine?

I am looking at the ProvideProfileAttribute now.

[aboimpinto]

Note that if you switch to ms code coverage then you need to use that method of exclusion - https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#include-or-exclude-assemblies-and-members

tkx for the answer ... Can you provide a complete example? I could not put the FCC work with the .runsettings file.

[aboimpinto]

For the exclusion of a file name pattern, I try to use the example in this discussion

https://github.com/FortuneN/FineCodeCoverage/discussions/298

and I add this to my csproj

but the settings in the output still the same from the settings page

how can the settings in the csproj overnight there ones in the settings?

[tonyhallett]

Can you provide a complete example? I could not put the FCC work with the .runsettings file.

Provide in visual studio options / the project file / fcc run settings file. Visit the link to understand how microsoft does exclusion and inclusion.

[aboimpinto]

But this article doesn't discuss excluding one file pattern but a whole DLLs ... which I can do already ... Now, I need to know how to exclude the files *.axaml.cs that is inside the DLL

[tonyhallett]

But this article doesn't discuss excluding one file pattern

Yes it does. Source, for which FCC has SourceExclude and SourceInclude.

[tonyhallett]

how can the settings in the csproj overnight there ones in the settings?

Have you added <PropertyGroup label="FineCodeCoverage"> in the project file ?

[aboimpinto]

how can the settings in the csproj overnight there ones in the settings?

Have you added <PropertyGroup label="FineCodeCoverage"> in the project file ?

yes ..

and the output in the console

[aboimpinto]

I still have the .runsettings file and be there or not it's the same ... maybe I need to do something else to make the build use it.

<?xml version="1.0" encoding="utf-8"?>

<RunSettings>
    <RunConfiguration>

        <TreatTestAdapterErrorsAsWarnings>false</TreatTestAdapterErrorsAsWarnings>
        <ResultsDirectory>.\TestResults</ResultsDirectory>

        <MaxCpuCount>0</MaxCpuCount>

        <TestSessionTimeout>10000</TestSessionTimeout>

        <TreatNoTestsAsError>false</TreatNoTestsAsError>

        <DisableParallelization>false</DisableParallelization>
        <DisableAppDomain>false</DisableAppDomain>
        <CollectSourceInformation>true</CollectSourceInformation>

        <BatchSize>10</BatchSize>

    </RunConfiguration>

    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="Code Coverage" uri="datacollector://Microsoft/CodeCoverage/2.0" assemblyQualifiedName="Microsoft.VisualStudio.Coverage.DynamicCoverageDataCollector, Microsoft.VisualStudio.TraceCollector, Version=12.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a">
                <Configuration>
                    <CodeCoverage>

                        <!-- Match assembly file paths: -->
                        <ModulePaths>
                            <Include>
                                <ModulePath>.*\.dll$</ModulePath>
                                <ModulePath>.*\.exe$</ModulePath>
                            </Include>

                            <Exclude />
                        </ModulePaths>

                        <!-- Match fully qualified names of functions: -->
                        <!-- (Use "\." to delimit namespaces in C# or Visual Basic, "::" in C++.)  -->
                        <Functions />

                        <!-- Match attributes on any code element: -->
                        <Attributes />

                        <!-- Match the path of the source files in which each method is defined: -->
                        <Sources>
                            <Exclude>
                                <Source>.*\\*.axaml.cs</Source>
                            </Exclude>
                        </Sources>

                        <!-- Match the company name property in the assembly: -->
                        <CompanyNames>
                            <Exclude>
                                <CompanyName>.*microsoft.*</CompanyName>
                            </Exclude>
                        </CompanyNames>

                        <!-- Match the public key token of a signed assembly: -->
                        <PublicKeyTokens />

                        <!-- We recommend you do not change the following values: -->
                        <UseVerifiableInstrumentation>True</UseVerifiableInstrumentation>
                        <AllowLowIntegrityProcesses>True</AllowLowIntegrityProcesses>
                        <CollectFromChildProcesses>True</CollectFromChildProcesses>
                        <CollectAspDotNet>False</CollectAspDotNet>

                    </CodeCoverage>
                </Configuration>
            </DataCollector>

        </DataCollectors>
    </DataCollectionRunSettings>

    <!-- MSTest adapter -->
    <MSTest>
        <MapInconclusiveToFailed>false</MapInconclusiveToFailed>
        <CaptureTraceOutput>true</CaptureTraceOutput>
        <DeleteDeploymentDirectoryAfterTestRunIsComplete>true</DeleteDeploymentDirectoryAfterTestRunIsComplete>
        <DeploymentEnabled>true</DeploymentEnabled>
        <DeployTestSourceDependencies>true</DeployTestSourceDependencies>
        <MapNotRunnableToFailed>false</MapNotRunnableToFailed>
        <EnableBaseClassTestMethodsFromOtherAssemblies>false</EnableBaseClassTestMethodsFromOtherAssemblies>

        <ForcedLegacyMode>false</ForcedLegacyMode>
    </MSTest>

</RunSettings>

I copied it from the link you provided... I was already there all day ...

[tonyhallett]

Have you added in the project file ?

Sorry, I meant have you added to the test project file ?

[tonyhallett]

I expect not as project file will overwrite that of visual studio options ( unless you specify merge - for an array setting )

[tonyhallett]

I still have the .runsettings file and be there or not it's the same ... maybe I need to do something else to make the build use it.

Yes, if you decide to provide your own then you need to tell visual studio to use it - https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2022#specify-a-run-settings-file-in-the-ide

Note that you do not need to provide one as FCC will create a runsettings file per test project using the settings that have been provided in visual studio options / the test project file / finecodecoverage-settings.xml.

[aboimpinto]

Well... I don't decide to provide my own ... I wonder if I need one or not ... (unfortunately, I'm doing this kind of DevOp work without motivation or knowledge.)

I want to be able to exclude some files inside a project (example *.axmal.cs) and some projects (like the UI Controls project)

Would you like me to have a .runsettings file? I have no idea ... please tell me if I need it and how I do it because I'm swimming in the mayonnaise here.

Thanks a lot Paulo Aboim Pinto

[tonyhallett]

You do not need to provide one.

You just need to understand the exclusion format of the code coverage provider that you choose to use and decide upon how you would like to tell FCC of your exclusion.

If you do not choose ms code coverage then you need to use glob patterns https://github.com/coverlet-coverage/coverlet/blob/master/Documentation/MSBuildIntegration.md#filters

The ExcludeByFile that you provided is correct ( although the * at the beginning could be )

The question is then, why did FCC not use this setting.

I have asked you if you added this in the test project file. So did you ?

If you choose to use ms code coverage then it is regular expressions https://learn.microsoft.com/en-us/visualstudio/test/customizing-code-coverage-analysis?view=vs-2022#regular-expressions

If you were to use SourcesExclude in either Visual Studio options, finecodecoverage-settings.xml or .csproj with

Then using

Include and exclude nodes use regular expressions, which aren't the same as wildcards. All matches are case-insensitive.

Some examples are:

.* matches a string of any characters

. matches a dot "."

( ) matches parentheses "( )"

\ matches a file path delimiter "\"

^ matches the start of the string

$ matches the end of the string

The regex matches any string of characters, the file path delimiter, then matches *.axaml.cs which is incorrect.

Try this instead, if you use ms code coverage, .*\\.*.axaml.cs$

[tonyhallett]

Can you also provide a way to export these settings to be consistent in every developer machine?

I have update the extension but the github action is failing. @FortuneN https://github.com/FortuneN/FineCodeCoverage/actions/runs/6429539943

The Personal Access Token used has expired.

[tonyhallett]

@aboimpinto If you want the exporting of FCC settings then you can use this release https://github.com/FortuneN/FineCodeCoverage/releases/tag/v1.1.195

[aboimpinto]

Thanks a lot for all the information ... I already tried to exclude specific files, and I was not adding that PropertyGroup in the Test Project. It's in now

and it's not been used

But, the file *.axaml.cs still appears in the report. the file

the report

Maybe the pattern in the PropertyGroup is not right ...

[aboimpinto]

I try with the patterns:

• */.axaml.cs
• */*.axaml.cs

The file is still shown in the report.

[aboimpinto]

in the main page of the project: https://github.com/FortuneN/FineCodeCoverage there is an example of the PropertyGroup

And the pattern in ExcludeByFile starts with a double star (**)

[tonyhallett]

When you click on the link, does the expected file open in visual studio ?

[aboimpinto]

Yes.. and that is the file I want to exclude

Before I added the tag [ExcudeFromCodeCoverage] and in a code review, I was requested/suggested to remove that from production code.

[tonyhallett]

If you switch to ms code coverage and use SourcesExclude .*\\.*.axaml.cs$ do you get the same result ?

[aboimpinto]

I have this in the .runsettings file

and the file is in the same level of the sln of the project. How the coverlet knows that should use this file?

[tonyhallett]

I want you to switch to ms code coverage, not coverlet. Do that in the visual studio options.

Do not provide the runsettings for now, let FCC do that. Add SourcesExclude. Do that in Vs options too for now.

[aboimpinto]

how I do that!??!? how to I change to MS Code Coverage?

[tonyhallett]

Open Vs options for Fine Code Coverage and set RunMsCodeCoverage to true. Then add to the SourcesExclude option.

[aboimpinto]

ok ...

and

and nothing in the report

[tonyhallett]

What does the Output Window FCC pane show ?

[aboimpinto]

ahhh the pattern was wrong ... I replace with

in the end after many File does not exist

and generates report and the excluded pattern still there

[tonyhallett]

Can you provide a minimal solution with a single avalonia view.

[aboimpinto]

yes ... can !!!

[tonyhallett]

With a test project please

[tonyhallett]

The only other thing that could be possible is that avalonia creates an additional Init2bView file and when we click the link it opens the .axaml.cs file.

[aboimpinto]

https://github.com/AboimPinto/BasicAvaloniaProject

I didn't add the FCC it's 100% clean

[tonyhallett]

Will have a look

[aboimpinto]

The only other thing that could be possible is that avalonia creates an additional Init2bView file and when we click the link it opens the .axaml.cs file.

Well .. if that is true ... I will have a justification for the "Code Auditors" and I will be able to justify have the tag [ExcludeFromCodeCoverage] in every view of the app

[tonyhallett]

It is true

[tonyhallett]

Exclude all .g.cs files and you are there

[tonyhallett]

in addition to the .axaml.cs files

[tonyhallett]

If you give your views in all your projects the same namespace part you could exclude by that instead of by source file extension.

[aboimpinto]

I had the exclude the .xaml files also ...

and now Not there ... YES!!!

and now ... the next step ... export this configuration in order to be used in all developer machines the same way!!

[tonyhallett]

Hide Moq and your test framework while you are there too !

[tonyhallett]

Just note that there are differences between ms code coverage and coverlet/opencover.
Ms code coverage only provides coverage for what you run in test explorer. The others run all tests and work by copying dlls and running tests again after the test explorer has.

Instead of exporting the vs settings you might be able to use finecodecoverage-settings.xml files

https://github.com/FortuneN/FineCodeCoverage#project-configuration

These are found by walking up the directory structure from the project directory. By applying the attribute topLevel='true' to the root element the walk stops.

These override ( or possibly merge ) visual studio options.

[aboimpinto]

I updated to the new version of FCC as you proposed.

Where should be the finecodecoverage-settings.xml file? At the root of the project?

[tonyhallett]

You can place it in a test project directory or above.
So you can add to a solution directory to be applicable to the solution.

If you had a repo directory with all your solutions you can place in there or an ascendant directory to apply to all solutions.

[aboimpinto]

I could add the finecodecoverage-settings.xml and now I could remove all the settings in FCC Settings Page and use the parameters I specified in the file.

You cracked this when you found that I had to exclude the *.g.cs. Thanks a lot ... I was around this for a long time with no solution ...

once again Thanks a lot ... you have been a great help Paulo Aboim Pinto