📞📱☎️📡🌐 Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
Is your feature request related to a problem? Please describe.
The YAML result log can aid compliance with regulatory requirements, ensuring traceability and accountability in case audits or reviews are necessary. Example of the information stored per test:
In case of a failed, skipped or n/a test, there is no details about the root cause or failed resources stored in the YAML file. The stdout/stderr must be looked up to obtain this information which is a problem in machine-to-machine communications.
Describe the solution you'd like
Add information in the YAML file for failed, skipped and n/a tests. Examples:
Failed test:
details: "Failed to increase replicas from 1 to 3"
exception: "ERROR -- cnf-testsuite: Pod scaling has timed-out"
details: "Users with cluster-admin RBAC permissions found"
failed_resources: ["ServiceAccount sonobuoy-serviceaccount"]
skipped test:
details: "node_drain chaos test requires the cluster to have at least two schedulable nodes"
n/a test:
details: "Pods are not using SELinux"
Remediation info needs not be captured in the YAML file.
Describe alternatives you've considered
Redirect the stdout/stderr output of the CNF testsuite to a file and parse the obtained data. This is ineffective as the stdout/stderr information does not seem to follow a common schema,
How will this be tested? aka Acceptance Criteria (optional)
(optional: unnecessary for things like some version bumps etc.)
Once this issue is addressed how will the fix be verified?
Additional context
Add any other context or screenshots about the feature request here.
Is your feature request related to a problem? Please describe. The YAML result log can aid compliance with regulatory requirements, ensuring traceability and accountability in case audits or reviews are necessary. Example of the information stored per test:
In case of a failed, skipped or n/a test, there is no details about the root cause or failed resources stored in the YAML file. The stdout/stderr must be looked up to obtain this information which is a problem in machine-to-machine communications.
Describe the solution you'd like Add information in the YAML file for failed, skipped and n/a tests. Examples:
Failed test:
skipped test:
details: "node_drain chaos test requires the cluster to have at least two schedulable nodes"
n/a test:
details: "Pods are not using SELinux"
Remediation info needs not be captured in the YAML file.
Describe alternatives you've considered Redirect the stdout/stderr output of the CNF testsuite to a file and parse the obtained data. This is ineffective as the stdout/stderr information does not seem to follow a common schema,
How will this be tested? aka Acceptance Criteria (optional)
(optional: unnecessary for things like some version bumps etc.)
Once this issue is addressed how will the fix be verified?
Additional context Add any other context or screenshots about the feature request here.