search

rockmelonqa / rockmelonqa-ide

Test Automation IDE that generates and runs Playwright and Selenium code
MIT License
4 stars 3 forks source link

Test case execution with multiple data scenarios #163

Open brogreenmoon opened 1 year ago

brogreenmoon commented 1 year ago

Execution

All data combinations for a test case are executed. Adjust the test case description in the "Run Tests" dialog to indicate the number of data scenarios that will be run:

image

veebs commented 1 year ago

@brogreenmoon - test cases do not have data sets, only test routines. You need to create a test case and then add a test routine and specificy which data set to run

The test runner only runs the selected test cases - there is no concept of data set for test runners.

Nothing to fix here - please close this ticket if you are happy with this answer.

brogreenmoon commented 1 year ago

nope, not quite happy yet jan. a test case can call multiple test routines with multiple datasets defined for each test routine, so there is a wider concept of data scenarios that applies to a test case, where a data scenario is a combination of test routine -level datasets that in totality is a dataset/data scenario that is used for one instance of the test case. this "data scenario" does not exist in terms of the underlying data model being persisted, but it is a key construct in terms of what is actually being executed, as per my comment in #161.

we're going to need to visualise this in #118 too - i think it's a bit silly if you can't actually see the data that applies to each executing instance of the test case - you have no idea what's going on otherwise, such as when a test case calls three test routines with 2, 3 and 5 datasets respectively. you need to be able to see the combinations that are in play in each instance, and in terms of this ticket it means identifying that 5 scenarios are in play and showing "5" to the user.

veebs commented 1 year ago

this is not going to be done in this milestone - no time and also it does not make sense for the currrent construct. Data sets are releated to test routines and not test cases. You can add more than 1 routine and with more than 1 data set for each to a test case. This means that any listing of routine and data set really does not make sense unless you see the context of the entire test case. To do this, you might as well open and view the test case file.

As I mentioned before, you you want to be granular about it, create a test case for each test routinue data set combo.

Moving out of this milestone.