search

cnti-testcatalog / testsuite

📞📱☎️📡🌐 Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
https://wiki.lfnetworking.org/display/LN/Test+Catalog
Apache License 2.0
174 stars 72 forks source link

[Feature] Cloud Native Tagging #849

Open wvwatson opened 3 years ago

wvwatson commented 3 years ago

Is your feature request related to a problem? Please describe. Different user profiles (Network equipment/software manufacturer, Telco Service Provider, CNF Developer, Cloud Native Community) have different non-functional requirements that they desire. The current system does not support a way to easily run the system with these concerns in mind.

Describe the solution you'd like The system should have different tags for the user profiles e.g. NEM, SP, or CN like so: cnf-testsuite nem cnf-testsuite sp cnf-testsuite cn

The documentation would have all tests that are not cloud native tests to have their own category under profile specific tests. This would allow the different user categories to contribute their own weighting.

Describe alternatives you've considered One alternative is to offer a la cart testing, which is to allow the user to list each test or category of tests specifically. Another alternative is to focus on having tests that measure cloud native non-functional requirements exclusively, allowing any networking concerns that fail 'cloud-nativeness' to fail outright.

kosstennbl commented 2 months ago

Don't see this as a big priority, but some additional test grouping can be implemented by creating separate SAM tasks, that would be running needed tests or whole test categories. To me - should be done if some request comes from different community (e.g. Silva), if they would benefit from something like this and wouldn't be able to easily implement this on their side. Doing this as precaution doesn't make much sense to me, we just don't know which tests would be useful for what users.