search

SpecFlowOSS / SpecFlow

#1 .NET BDD Framework. SpecFlow automates your testing & works with your existing code. Find Bugs before they happen. Behavior Driven Development helps developers, testers, and business representatives to get a better understanding of their collaboration
https://www.specflow.org/
Other
2.24k stars 754 forks source link

How to run specific iterations in scenarios? #2530

Open alexandermikuta opened 2 years ago

alexandermikuta commented 2 years ago

Hi,

how is it possible to run only specific iterations of a scenario? In the documenation I could not find anything about this scenario.

Examples: | Example | | 1 | | 2 | | 3 | | 4 |

Here it should be possible to start the scenario only with iterations e.g. 2,3 or 1,3...

In https://github.com/SpecFlowOSS/SpecFlow/issues/1352 only filters were mentioned as an idea for a workaround. But that would mean that every iteration would need its own tag to be able to execute all possible iteration-combinations via command-line. Is there no easy way trigger this via command-line by e.g. using the index of the iterations?

e.g. <some commandline-call to trigger test> --iterations=2,3

Kind regards, Alexander

shack05 commented 2 years ago

Hi @alexandermikuta,

Are you using NUnit/MSTest/xUnit or the SpecFlow+ Runner, and what executable do you expect to use to run the tests from the command line?

I am not familiar with the SpecFlow+ Runner so I am unable to speak to that.

If you are using NUnit/MSTest/xUnit I am not aware of a 'simple' way to achieve what you want, however a subset of tests can be executed from the command line using dotnet test or vstest.console.exe by providing a filter (as described here). The filter language is quite extensive and although it can achieve what you are after I feel the resulting filter expressions can get quite long. Perhaps writing a script that can build the required filter expression and then invoke dotnet test or vstest could save some time in the long run.

Here's an example. Given the following tests: image

Assuming you have a recent version of visual studio and/or the dotnet cli installed, you can run all tests by opening up a command prompt in the same directory as the SpecFlowDemo.xUnit project file, and running:

dotnet test --no-build

--no-build isn't necessary, it just saves some time if the project doesn't need to be rebuilt before running the tests

To run the first example row:

dotnet test --no-build --filter "DisplayName~firstNumber: %2250"

That filter is saying: select all tests whose display name contains 'firstNumber: "50'

Note, %22 is a url encoded double quote.

To run the first and second example rows:

dotnet test --no-build --filter "DisplayName~firstNumber: %2250|DisplayName~firstNumber: %221"

More information about selectively running tests is available here.

dotnet test includes a --list-tests option, although no tests were discovered for me when running the above commands and appending --list-tests.