Support multiple --filter-test-case/--filter-test-list flags

[pbiggar]

I have found it challenging to isolate individual tests. For example, it's hard to isolate A.B.C.D.e when there are test cases also named e and test lists also named A, B, C, D or e.

If I could filter on the entire test name, or add multiple filters (which would all need to be true), I'd have really powerful ability to run just one test.

I've run into this problem I would say hundreds of times over the last year or two.

[ratsclub]

Hello, @pbiggar. May you please provide a repository with the code and perhaps the expected output?

[pbiggar]

I don't have a particular implementation in mind, but one way would be to allow multiple --filter calls to imply an and relationship.

So for example, with the darklang codebase I want to run:

./Tests --filter-test-list float --filter-test-case line5

which should match tests like:

tests.LibExecution.All.float.line5 starting
tests.LibExecution.All.float.line58 starting
tests.LibExecution.All.float.line59 starting
tests.LibExecution.All.float.line50 starting

but not tests that don't have both "line5" in the test name and "float" in the name of a test list.

Another useful way to approach this would be to allow filtering on "fully-qualified" test names such as "float.line5" or "All.*.line5"

[baronfel]

There's also prior art for nunit/mstest/etc to accept a small DSL for test case selection in the 'filter' parameter. There are docs on this here: https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests?pivots=mstest

[farlee2121]

YoloDev added support for --filter use with dotnet test.

The docs are a bit sparse, but filters are in this general shape ((FullyQualifiedName~TestContainsThis) & (FullyQualifiedName~alsoContainsThis)) | (FullyQualifiedName=Or.Exact.Name.Here)