search

dotnet / BenchmarkDotNet

Powerful .NET library for benchmarking
https://benchmarkdotnet.org
MIT License
10.53k stars 968 forks source link

Unable to filter benchmarks with TestAdapter #2662

Open olstakh opened 2 hours ago

olstakh commented 2 hours ago

Steps to reproduce

  1. Follow https://benchmarkdotnet.org/articles/features/vstest.html (console project + TestAdapter package + Test.Sdk)
  2. Define 2 benchmarks with different [BenchmarkCategory("MyTest1")] and [BenchmarkCategory("MyTest2")] attributes
  3. Run dotnet test -c Release --filter "Category=MyTest1"

Expected: 1 benchmark executed (category attribute is being added as a trait )

Actual: both benchmarks executed

Also [unrelated] - are these benchmarks supposed to show up in Test Explorer in VS? I'd assume they would, since dotnet test picks them up

timcassell commented 2 hours ago

Is that filter behavior the same without the TestAdapter?

cc @caaavik-msft