search

kubeshop / testkube

☸️ Kubernetes-native testing framework for test execution and orchestration
https://testkube.io
Other
1.33k stars 130 forks source link

kubectl testkube create multiple tests from certain path #2017

Open hosumihaidarius opened 2 years ago

hosumihaidarius commented 2 years ago

This feature request is to easily manage the creation of multiple tests when using the kubectl testkube create test command.

If we work under the assumption that multiple teams must create multiple tests ( not necessarily in suites ) using different runners, a method that identifies all e.g. : k6 scripts from within a certain directory, or cypress tests from a directory would be beneficial.

It would also be nice if we were allowed for example that upon creating multiple tests using kubectl create test , the test names to be mapped by default as the test file.

e.g. for k6 : tests load syntheticTestConstantRPS.js heavyLoadTestS3Bucket.js stressTestSomeService.js

using : kubectl testkube create tests --dir tests/load/*.js --type "k6/custom" --names **optional args for test names in array like fashion ["Synthetic test with constant rps", "Heavy load test on S3 Buckets", "Stress test for some service"] or if left blank, just pick up the name of the js files.

olensmar commented 2 years ago

nice! - how does this relate to https://github.com/kubeshop/testkube/issues/1277 ? would 1277 be an extension to this feature?

exu commented 2 years ago

looks like the sub-issue of our scanner would work nicely as a good starting point. One issue which I see is about this name array It'll be quite hard to define it easily as he would need to know all possible tests. I think we could just use the file/dir (user could just name them correctly)

hosumihaidarius commented 2 years ago

in #1277 there is also the statement that it will look for different kinds of tests. how will this map in circumstances in which we create our own custom executors ?

e.g. we currently used the templates provided by testkube to create a custom runner with xk6. I'm guessing it will be pretty difficult to identify all tests, map them to the default testkube runners but some of them to be mapped with a certain custom type e.g. --type "k6/custom"

olensmar commented 2 years ago

good point @hosumihaidarius - 1277 is mainly to help (new) users to onboard Testkube using existing tests with default executors - for more advanced scenarios we would need to provide some configuration possibilities to allow for custom executors, file-matching, naming, etc.

TheBrunoLopes commented 1 year ago

Closing this issue as duplicated. Please track here: https://github.com/kubeshop/testkube/issues/1277