cnti-testcatalog / testsuite

📞📱☎️📡🌐 Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
https://wiki.lfnetworking.org/display/LN/Test+Catalog
Apache License 2.0
169 stars 70 forks source link

[MAINTENACE] Clarify usage and behavior of testing multiple CNFs #2095

Open martin-mat opened 4 days ago

martin-mat commented 4 days ago

Currently, it is possible to run cnf_setup multiple times for multiple CNFs/helm charts. When running tests then, tests are executed on each configured CNFs. Clarify whether this is a supported feature. Currently, there is no documentation nor tests of this behavior. If this should be supported behavior, address those aspects:

  1. Add this to documentation
  2. Add spec test(s) to verify expected functionality
  3. Fix behavior of generating result yaml files. Currnently, just one yaml file is being generated; results of particular test are overwritten, and it records result only of the latest CNF. Which if obviously not expected behavior. Right behavior needs to be figured out and implemented.

If this is not supported usage, don't allow to call cnf_setup multiple times.

martin-mat commented 4 days ago

2090

martin-mat commented 4 days ago

My input for the discussion/decision:

I propose not to have support for this. Reasons:

Smitholi67 commented 3 days ago

+1 for not supporting - let us keep it simple.