cnti-testcatalog / testsuite

πŸ“žπŸ“±β˜ŽοΈπŸ“‘πŸŒ Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
https://wiki.lfnetworking.org/display/LN/Test+Catalog
Apache License 2.0
173 stars 71 forks source link

[Feature] Add test case identifiers for tests being executed to console output and more informative console output #1958

Closed martin-mat closed 4 months ago

martin-mat commented 5 months ago

Is your feature request related to a problem? Please describe. Currently, console output does not provide any information about what testcase is being executed/has been executed. Example:

  1. start test execution

$ ./cnf-testsuite single_process_type

(test is being executed, user is not informed about any action ongoing)

  1. tests finishes

$ ./cnf-testsuite single_process_type βœ”οΈ πŸ†PASSED: Only one process type used βš–πŸ‘€ $

(test execution finished, user is informed about the test result, but no explicit test identifier (single_process_type) given. User needs to guess what test case it was (from the result note "Only one process type used")

Describe the solution you'd like Give more information about what is going on and an exact test description. Example:

  1. start test execution

$ ./cnf-testsuite single_process_type 🎬 Testing "single_process_type"

(test is being executed, user is not informed about any action ongoing)

  1. tests finishes

$ ./cnf-testsuite single_process_type 🎬 Testing: single_process_type βœ”οΈ πŸ†PASSED: single_process_type βš–πŸ‘€ $

or

$ ./cnf-testsuite single_process_type 🎬 Testing: single_process_type βœ–οΈ πŸ†FAILED: single_process_type βš–πŸ‘€ $

Eventually, the "Testing" line can be replaced on then console with the result line after the test is executed. But that is probably not feasible now as the console output is mixed with logs.

Where needed, keep additional information for failed tests (where it is now)

Note that this is related to "console output". Increasing log level is not a solution for this as logging is (or rather should be) a different mechanism with potentially routing to different files than the console output.

HashNuke commented 5 months ago

Yes this would help ~!

HashNuke commented 5 months ago

+1 for the "Testing" line.

martin-mat commented 5 months ago

Yes this would help ~!

  • We can prefix it "PASSED: [test name] Hello world"
  • Or we can suffix it to the end of the message Like "PASSED: Hello world (test name)"

"PASSED: [test name] Hello world" looks good.