eykrehbein / strest

⚡️ CI-ready tests for REST APIs configured in YAML
MIT License
1.74k stars 59 forks source link

execution order #80

Closed jgroom33 closed 6 years ago

jgroom33 commented 6 years ago

execution order should be:

test_dir A_subdir 3.strest.yml 4.strest.yml Z_subdir 5.strest.yml 6.strest.yml 1.strest.yml 2.strest.yml

Executing strest tests/success/ results in the following execution order:

✔ Testing todoOne succeeded (0.188s)
✔ Testing todoTwo succeeded (0.107s)
✔ Testing todoOne succeeded (0.221s)
✔ Testing todoTwo succeeded (0.102s)
✔ Testing postman succeeded (0.293s)
✔ Testing my_variable_request succeeded (0.102s)
✔ Testing environment succeeded (0.101s)
✔ Testing fake succeeded (0.195s)
✔ Testing arr succeeded (0.571s)
✔ Testing arr1 succeeded (0.106s)
✔ Testing value1 succeeded (0.215s)
✔ Testing value2 succeeded (0.184s)
✔ Testing basic_auth succeeded (0.19s)
✔ Testing logging succeeded (0.197s)
✔ Testing if_Set succeeded (0.372s)
✔ Testing skipped skipped (0.001s)
✔ Testing executed succeeded (0.105s)
✔ Testing codeValidate succeeded (0.102s)
✔ Testing code404 succeeded (0.671s)
✔ Testing headersValidate succeeded (0.124s)
✔ Testing jsonValidate succeeded (0.1s)
✔ Testing jsonpath succeeded (0.264s)
✖ Testing maxRetries failed to validate. Retrying... (1.192s)
✔ Testing maxRetries succeeded (0.193s)
✔ Testing raw succeeded (0.191s)
✔ Testing login succeeded (0.188s)
✔ Testing verify_login succeeded (0.41s)
✔ Testing verify_login_chained succeeded (0.282s)
✔ Testing chaining_var1 succeeded (0.096s)
✔ Testing chaining_var2 succeeded (0.108s)

Based on current tests, this should be the order:

✔ Testing todoOne succeeded (0.188s)
✔ Testing todoTwo succeeded (0.107s)
✔ Testing postman succeeded (0.293s)
✔ Testing todoOne succeeded (0.221s)
✔ Testing todoTwo succeeded (0.102s)
✔ Testing my_variable_request succeeded (0.102s)
✔ Testing basic_auth succeeded (0.19s)
✔ Testing login succeeded (0.188s)
✔ Testing verify_login succeeded (0.41s)
✔ Testing verify_login_chained succeeded (0.282s)
✔ Testing chaining_var1 succeeded (0.096s)
✔ Testing chaining_var2 succeeded (0.108s)