Reworked modules filtering process

[BlackGad]

Recently, we encountered a performance issue with Coverlet during our CI pipeline processes.

Before executing dotnet test, we preprocess our CI build files to generate a runsettings file. This file specifies which assemblies to include and exclude based on pattern matching.

The output directory ends up containing over 350 assemblies, including framework ones. Within this, the include section lists 54 patterns in the format *[Assembly1],[Assembly2], while the exclude section contains 301** patterns, also specified in a similar manner.

Runsettings example:

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
    <DataCollectionRunSettings>
        <DataCollectors>
            <DataCollector friendlyName="XPlat code coverage">
                <Configuration>
                    <Exclude>[AutoFixture.AutoMoq]*,[AutoFixture]*,[AutoFixture.Xunit2]*,[AutoMapper]*,[AutoMapper.Extensions.ExpressionMapping]*,[AWSSDK.Core]*,[AWSSDK.SecurityToken]*,[Azure.Core.Amqp]*,[Azure.Core]*,[Azure.Data.AppConfiguration]*,[Azure.Extensions.AspNetCore.Configuration.Secrets]*,[Azure.Extensions.AspNetCore.DataProtection.Blobs]*,[Azure.Extensions.AspNetCore.DataProtection.Keys]*,[Azure.Identity]*,[Azure.Messaging.EventGrid]*,[Azure.Messaging.EventHubs]*,[Azure.Messaging.EventHubs.Processor]*,[Azure.Messaging.ServiceBus]*,[Azure.Security.KeyVault.Certificates]*,[Azure.Security.KeyVault.Keys]*,[Azure.Security.KeyVault.Secrets]*,[Azure.Storage.Blobs]*,[Azure.Storage.Common]*,[Azure.Storage.Files.Shares]*,[Azure.Storage.Queues]*,[Castle.Core]*,[CliWrap]*,[CommandLine]*,[coverlet.collector]*,[coverlet.core]*,[BlackGad.Cache]*,[BlackGad.Cache.Redis.Tests]*,[BlackGad.Client]*,[BlackGad.Client.Operation.Http.TwoWay.Tests]*,[BlackGad.Client.Operation.MessageBus.OneWay.Tests]*,[BlackGad.Client.Proxy.Provider.Tests]*,[BlackGad.ClientBootstrapper.Tests]*,[BlackGad.Common]*,[BlackGad.Composition]*,[BlackGad.Configuration.Check.Tests]*,[BlackGad.Configuration]*,[BlackGad.DataEncryption]*,[BlackGad.DataEncryption.Providers.Tests]*,[BlackGad.DiagnosticsFramework.AppInsights.Tests]*,[BlackGad.DiagnosticsFramework]*,[BlackGad.DiagnosticUtility.Tests]*,[BlackGad.DistributedLock]*,[BlackGad.DistributedLock.Redis.Tests]*,[BlackGad.Metrics.Prometheus.Tests]*,[BlackGad.EventHub.Azure.Tests]*,[BlackGad.EventHub]*,[BlackGad.Filters]*,[BlackGad.Filters.InMemory.Tests]*,[BlackGad.Filters.Mongo.Tests]*,[BlackGad.Host.LeaderFollower]*,[BlackGad.Localization]*,[BlackGad.Logging.AppInsights.Tests]*,[BlackGad.MBus.ServiceBus.Tests]*,[BlackGad.MessageBus]*,[BlackGad.ObjectEventProvider]*,[BlackGad.ObjectEventProvider.PubSub.Tests]*,[BlackGad.OData.Tests]*,[BlackGad.PostDeploymentTool]*,[BlackGad.PostDeploymentTool.Executor.Tests]*,[BlackGad.Redis.Tests]*,[BlackGad.Repository.Mongo.Tests]*,[BlackGad.Schedule]*,[BlackGad.Schedule.Quartz.Tests]*,[BlackGad.ScheduleHost.Tests]*,[BlackGad.SecureDataProvider.AzureKeyVault.Tests]*,[BlackGad.SecureDataProvider]*,[BlackGad.Security.AA.Tests]*,[BlackGad.Security]*,[BlackGad.Serializer.Json.Tests]*,[BlackGad.Service]*,[BlackGad.Service.WebApi.Tests]*,[BlackGad.ServiceDiscoveryApiClient.Tests]*,[BlackGad.ServiceHost.Tests]*,[BlackGad.Storage]*,[BlackGad.Storage.Sql.Tests]*,[BlackGad.Stream.AzureBlob.Tests]*,[BlackGad.Stream]*,[BlackGad.Web.Tests]*,[BlackGad.WebNotifications]*,[BlackGad.WebNotifications.SignalR.Tests]*,[DnsClient]*,[dura]*,[Fare]*,[FluentAssertions]*,[Flurl]*,[Fractions]*,[Google.Protobuf]*,[Grpc.AspNetCore.Server.ClientFactory]*,[Grpc.AspNetCore.Server]*,[Grpc.Core.Api]*,[Grpc.Net.Client]*,[Grpc.Net.ClientFactory]*,[Grpc.Net.Common]*,[HealthChecks.AzureStorage]*,[HealthChecks.MongoDb]*,[HealthChecks.OpenIdConnectServer]*,[HealthChecks.Redis]*,[HealthChecks.Uris]*,[IdentityModel.AspNetCore.OAuth2Introspection]*,[IdentityModel]*,[IdentityModel.OidcClient]*,[IdentityServer4.AccessTokenValidation]*,[KubernetesClient.Basic]*,[KubernetesClient]*,[KubernetesClient.Models]*,[Microsoft.AI.DependencyCollector]*,[Microsoft.AI.EventCounterCollector]*,[Microsoft.AI.PerfCounterCollector]*,[Microsoft.AI.ServerTelemetryChannel]*,[Microsoft.AI.WindowsServer]*,[Microsoft.ApplicationInsights.AspNetCore]*,[Microsoft.ApplicationInsights]*,[Microsoft.ApplicationInsights.Kubernetes]*,[Microsoft.AspNetCore.Authentication.JwtBearer]*,[Microsoft.AspNetCore.Connections.Abstractions]*,[Microsoft.AspNetCore.Cryptography.Internal]*,[Microsoft.AspNetCore.DataProtection.Abstractions]*,[Microsoft.AspNetCore.DataProtection]*,[Microsoft.AspNetCore.Http.Connections.Client]*,[Microsoft.AspNetCore.Http.Connections.Common]*,[Microsoft.AspNetCore.JsonPatch]*,[Microsoft.AspNetCore.Mvc.NewtonsoftJson]*,[Microsoft.AspNetCore.Mvc.Testing]*,[Microsoft.AspNetCore.Mvc.Versioning]*,[Microsoft.AspNetCore.OData]*,[Microsoft.AspNetCore.SignalR.Client.Core]*,[Microsoft.AspNetCore.SignalR.Client]*,[Microsoft.AspNetCore.SignalR.Common]*,[Microsoft.AspNetCore.SignalR.Protocols.Json]*,[Microsoft.AspNetCore.TestHost]*,[Microsoft.Azure.Amqp]*,[Microsoft.Azure.AppConfiguration.AspNetCore]*,[Microsoft.Azure.Cosmos.Table]*,[Microsoft.Azure.DocumentDB.Core]*,[Microsoft.Azure.Services.AppAuthentication]*,[Microsoft.Bcl.AsyncInterfaces]*,[Microsoft.CodeCoverage.Core]*,[Microsoft.CodeCoverage.Instrumentation]*,[Microsoft.CodeCoverage.Interprocess]*,[Microsoft.Data.SqlClient]*,[Microsoft.Data.Sqlite]*,[Microsoft.DiaSymReader]*,[Microsoft.EntityFrameworkCore.Abstractions]*,[Microsoft.EntityFrameworkCore]*,[Microsoft.EntityFrameworkCore.InMemory]*,[Microsoft.EntityFrameworkCore.Relational]*,[Microsoft.EntityFrameworkCore.SqlServer]*,[Microsoft.Extensions.Caching.Abstractions]*,[Microsoft.Extensions.Caching.Memory]*,[Microsoft.Extensions.Configuration.Abstractions]*,[Microsoft.Extensions.Configuration.AzureAppConfiguration]*,[Microsoft.Extensions.Configuration.Binder]*,[Microsoft.Extensions.Configuration.CommandLine]*,[Microsoft.Extensions.Configuration]*,[Microsoft.Extensions.Configuration.EnvironmentVariables]*,[Microsoft.Extensions.Configuration.FileExtensions]*,[Microsoft.Extensions.Configuration.Json]*,[Microsoft.Extensions.Configuration.UserSecrets]*,[Microsoft.Extensions.DependencyInjection.Abstractions]*,[Microsoft.Extensions.DependencyInjection]*,[Microsoft.Extensions.DependencyModel]*,[Microsoft.Extensions.DiagnosticAdapter]*,[Microsoft.Extensions.Diagnostics.Abstractions]*,[Microsoft.Extensions.Diagnostics]*,[Microsoft.Extensions.Diagnostics.HealthChecks.Abstractions]*,[Microsoft.Extensions.Diagnostics.HealthChecks]*,[Microsoft.Extensions.Diagnostics.HealthChecks.EntityFrameworkCore]*,[Microsoft.Extensions.Features]*,[Microsoft.Extensions.FileProviders.Abstractions]*,[Microsoft.Extensions.FileProviders.Physical]*,[Microsoft.Extensions.FileSystemGlobbing]*,[Microsoft.Extensions.Hosting.Abstractions]*,[Microsoft.Extensions.Hosting]*,[Microsoft.Extensions.Http.Polly]*,[Microsoft.Extensions.Localization.Abstractions]*,[Microsoft.Extensions.Localization]*,[Microsoft.Extensions.Logging.Abstractions]*,[Microsoft.Extensions.Logging.ApplicationInsights]*,[Microsoft.Extensions.Logging.Configuration]*,[Microsoft.Extensions.Logging.Console]*,[Microsoft.Extensions.Logging.Debug]*,[Microsoft.Extensions.Logging]*,[Microsoft.Extensions.Logging.EventLog]*,[Microsoft.Extensions.Logging.EventSource]*,[Microsoft.Extensions.Options.ConfigurationExtensions]*,[Microsoft.Extensions.Options]*,[Microsoft.Extensions.Primitives]*,[Microsoft.Identity.Client]*,[Microsoft.Identity.Client.Extensions.Msal]*,[Microsoft.IdentityModel.Abstractions]*,[Microsoft.IdentityModel.Clients.ActiveDirectory]*,[Microsoft.IdentityModel.JsonWebTokens]*,[Microsoft.IdentityModel.Logging]*,[Microsoft.IdentityModel.Protocols]*,[Microsoft.IdentityModel.Protocols.OpenIdConnect]*,[Microsoft.IdentityModel.Tokens]*,[Microsoft.OData.Client]*,[Microsoft.OData.Core]*,[Microsoft.OData.Edm]*,[Microsoft.OData.ModelBuilder]*,[Microsoft.OpenApi]*,[Microsoft.Spatial]*,[Microsoft.SqlServer.Server]*,[Microsoft.TestPlatform.CommunicationUtilities]*,[Microsoft.TestPlatform.CoreUtilities]*,[Microsoft.TestPlatform.CrossPlatEngine]*,[Microsoft.TestPlatform.PlatformAbstractions]*,[Microsoft.TestPlatform.Utilities]*,[Microsoft.VisualStudio.CodeCoverage.Shim]*,[Microsoft.VisualStudio.TestPlatform.Common]*,[Microsoft.VisualStudio.TestPlatform.ObjectModel]*,[Microsoft.VisualStudio.TraceDataCollector]*,[Microsoft.Win32.SystemEvents]*,[MongoDB.Bson]*,[MongoDB.Driver.Core]*,[MongoDB.Driver]*,[MongoDB.Libmongocrypt]*,[Mono.Cecil]*,[Mono.Cecil.Mdb]*,[Mono.Cecil.Pdb]*,[Mono.Cecil.Rocks]*,[Moq]*,[Moq.ILogger]*,[Newtonsoft.Json.Bson]*,[Newtonsoft.Json]*,[NuGet.Frameworks]*,[NuGet.Versioning]*,[Pipelines.Sockets.Unofficial]*,[Polly]*,[Polly.Extensions.Http]*,[Prometheus.AspNetCore]*,[Prometheus.NetStandard]*,[Quartz]*,[Quartz.Jobs]*,[Quartz.Plugins]*,[Quartz.Plugins.TimeZoneConverter]*,[Quartz.Serialization.Json]*,[RichardSzalay.MockHttp]*,[Serilog.AspNetCore]*,[Serilog]*,[Serilog.Extensions.Hosting]*,[Serilog.Extensions.Logging]*,[Serilog.Formatting.Compact]*,[Serilog.Settings.Configuration]*,[Serilog.Sinks.ApplicationInsights]*,[Serilog.Sinks.Console]*,[Serilog.Sinks.Debug]*,[Serilog.Sinks.File]*,[SharpCompress]*,[Snappier]*,[SQLitePCLRaw.batteries_v2]*,[SQLitePCLRaw.core]*,[SQLitePCLRaw.provider.e_sqlite3]*,[StackExchange.Redis]*,[Swashbuckle.AspNetCore.Annotations]*,[Swashbuckle.AspNetCore.Newtonsoft]*,[Swashbuckle.AspNetCore.Swagger]*,[Swashbuckle.AspNetCore.SwaggerGen]*,[Swashbuckle.AspNetCore.SwaggerUI]*,[System.Buffers]*,[System.Collections.Immutable]*,[System.ComponentModel.Composition]*,[System.Configuration.ConfigurationManager]*,[System.Diagnostics.EventLog]*,[System.Diagnostics.PerformanceCounter]*,[System.Drawing.Common]*,[System.IdentityModel.Tokens.Jwt]*,[System.IO.Hashing]*,[System.IO.Pipelines]*,[System.Linq.Dynamic.Core]*,[System.Memory.Data]*,[System.Memory]*,[System.Numerics.Vectors]*,[System.Reflection.Metadata]*,[System.Reflection.MetadataLoadContext]*,[System.Runtime.Caching]*,[System.Runtime.CompilerServices.Unsafe]*,[System.Security.Cryptography.ProtectedData]*,[System.Security.Permissions]*,[System.Text.Encodings.Web]*,[System.Text.Json]*,[System.Threading.Tasks.Extensions]*,[System.Windows.Extensions]*,[testhost]*,[TimeZoneConverter]*,[xunit.abstractions]*,[xunit.assert]*,[xunit.core]*,[xunit.execution.dotnet]*,[xunit.runner.reporters.netcoreapp10]*,[xunit.runner.utility.netcoreapp10]*,[xunit.runner.visualstudio.testadapter]*,[YamlDotNet]*,[ZstdSharp]*,[dura]*,[testhost]*,</Exclude>
                    <Include>[BlackGad.Cache.Redis]*,[BlackGad.Client.Operation.Http.TwoWay]*,[BlackGad.Client.Operation.MessageBus.OneWay]*,[BlackGad.Client.Proxy.Provider]*,[BlackGad.ClientBootstrapper]*,[BlackGad.Composition.MEF]*,[BlackGad.Configuration.Check]*,[BlackGad.DataEncryption.Providers]*,[BlackGad.DiagnosticsFramework.AppInsights]*,[BlackGad.DiagnosticsFramework.PlatformProcessors]*,[BlackGad.DiagnosticsFramework.WorkbookGenerator]*,[BlackGad.DistributedLock.Redis]*,[BlackGad.Metrics.Prometheus]*,[BlackGad.EventHub.Azure]*,[BlackGad.Filters.InMemory]*,[BlackGad.Filters.Mongo]*,[BlackGad.Filters.Sql]*,[BlackGad.Host.LeaderFollower.Provider]*,[BlackGad.Localization.Providers]*,[BlackGad.Logging.AppInsights]*,[BlackGad.MBus.ServiceBus.Reliability]*,[BlackGad.MBus.ServiceBus]*,[BlackGad.MongoDb.Telemetry]*,[BlackGad.ObjectEventProvider.PubSub]*,[BlackGad.OData]*,[BlackGad.PostDeploymentTool.Executor]*,[BlackGad.Redis]*,[BlackGad.Repository.Mongo]*,[BlackGad.Schedule.Quartz]*,[BlackGad.ScheduleHost]*,[BlackGad.SecureDataProvider.AzureKeyVault]*,[BlackGad.Security.AA]*,[BlackGad.Serializer.Json]*,[BlackGad.Service.Autoscale.Keda]*,[BlackGad.Service.Autoscale.PlatformCalculators]*,[BlackGad.Service.WebApi]*,[BlackGad.ServiceDiscoveryApiClient]*,[BlackGad.ServiceHost]*,[BlackGad.Storage.MongoDb]*,[BlackGad.Storage.Sql]*,[BlackGad.Stream.AzureBlob]*,[BlackGad.Web]*,[BlackGad.WebApiHost]*,[BlackGad.WebNotifications.SignalR]*,[BlackGad.DiagnosticUtility.Common]*,[BlackGad.DiagnosticUtilityResourceAnalyzer]*,[BlackGad.PostDeploymentTool.Executor.Example]*,[BlackGad.Sample.TestModule.ConsoleClient]*,[BlackGad.Sample.TestModule.WebApiClient]*,[BlackGad.Sample.TestModuleHost.Services]*,[BlackGad.Sample.TestModuleHost]*,[BlackGad.Sample.TestModuleSDK]*,[BlackGad.Sample.TestScheduleHost]*,[BlackGad.Client.Proxy.Provider.Tests.SDK]*,[BlackGad.Tests.Common]*,</Include>
                    <IncludeDirectory>_publish</IncludeDirectory>
                    <SkipAutoProps>true</SkipAutoProps>
                    <DeterministicReport>true</DeterministicReport>
                </Configuration>
            </DataCollector>
        </DataCollectors>
    </DataCollectionRunSettings>
</RunSettings>

The initial step of the testing operation involves Coverlet filtering the modules it needs to instrument. Using the aforementioned runsettings for filtering resulted in a significant delay of 50 seconds, primarily due to the inefficient implementation of the include and exclude filters. This Pull Request addresses and resolves this issue, reducing the filtering time to just 200 milliseconds.

Although I believe there's room for further optimization, particularly with type filtering, I consider that an objective for future development. I respectfully request a review and approval for merging this PR.

closes #1646

[BlackGad]

Here the code example with comparison https://github.com/BlackGad/Coverlet.Issue

[daveMueller]

/azp run

[azure-pipelines[bot]]

Azure Pipelines successfully started running 1 pipeline(s).

[daveMueller]

I looked at the PR and think this is good. Thanks a lot @BlackGad 🙏 . I only refactored some minor things to increase readability and will see if I can add some more tests. Will push my changes somewhere next week and then we can merge.

[BlackGad]

Thank you! Please do it asap) Kinda very annoying issue :)

[BlackGad]

Can't wait to see release version to test. Thank you for merging!

[daveMueller]

@BlackGad You can already consume this by referencing our nightly build here.