[Documentation] create List of Tests file for v1.1 beta

[lixuna]

[Documentation] create List of Tests file for v1.1 beta

Short description:

• provide a summary of the CNF Certification tests included in v1.1 beta release
• Example: https://github.com/cncf/cnf-certification/blob/main/docs/CNFCertification-1.0-beta.md

Tasks:

• create new file for 1.1-beta.md
• copy/paste content from 1.0-beta.md
• modify content with changes for 1.1-beta.md
• etc

[agentpoyo]

1.1-beta.md has been created. Need to verify list of tests in this file with the tests that run during cert command.

[agentpoyo]

Here are the list of tests from results output when running cert:

- name: helm_chart_valid
- name: helm_chart_published
- name: helm_deploy
- name: increase_decrease_capacity
- name: rollback
- name: no_local_volume_configuration
- name: elastic_volumes
- name: node_drain
- name: symlink_file_system
- name: privilege_escalation
- name: insecure_capabilities
- name: resource_policies
- name: linux_hardening
- name: ingress_egress_blocked
- name: host_pid_ipc_privileges
- name: non_root_containers
- name: privileged_containers
- name: immutable_file_systems
- name: hostpath_mounts
- name: container_sock_mounts
- name: external_ips
- name: selinux_options
- name: sysctls
- name: host_network
- name: service_account_mapping
- name: application_credentials
- name: nodeport_not_used
- name: hostport_not_used
- name: hardcoded_ip_addresses_in_k8s_runtime_configuration
- name: secrets_used
- name: immutable_configmap
- name: require_labels
- name: latest_tag
- name: default_namespace
- name: log_output
- name: prometheus_traffic
- name: open_metrics
- name: routed_logs
- name: tracing
- name: reasonable_image_size
- name: reasonable_startup_time
- name: single_process_type
- name: service_discovery
- name: shared_database
- name: pod_network_latency
- name: pod_network_corruption
- name: disk_fill
- name: pod_delete
- name: pod_memory_hog
- name: pod_io_stress
- name: pod_dns_error
- name: pod_network_duplication
- name: liveness
- name: readiness

[agentpoyo]

Here are the list of tests from embedded points.yml file that are marked with the cert tag:

reasonable_image_size
reasonable_startup_time
single_process_type
service_discovery
shared_database
cni_compatible
privilege_escalation
symlink_file_system
application_credentials
host_network
increase_decrease_capacity
pod_network_latency
pod_network_corruption
pod_network_duplication
pod_delete
pod_io_stress
pod_memory_hog
disk_fill
pod_dns_error
liveness
readiness
rollback
nodeport_not_used
hostport_not_used
hardcoded_ip_addresses_in_k8s_runtime_configuration
secrets_used
immutable_configmap
helm_deploy
helm_chart_valid
helm_chart_published
no_local_volume_configuration
elastic_volumes
node_drain
service_account_mapping
privileged_containers
non_root_containers
host_pid_ipc_privileges
linux_hardening
resource_policies
immutable_file_systems
hostpath_mounts
ingress_egress_blocked
insecure_capabilities
sysctls
log_output
prometheus_traffic
open_metrics
routed_logs
tracing
container_sock_mounts
require_labels
external_ips
selinux_options
default_namespace
latest_tag

[agentpoyo]

The 55 count is accurate. My actual cert run did not log the cni_compatible test as it skipped for some reason, which might be a bug as it should still add it to the results log. I'll verify this behavior to reproduce if possible to address.

The list of tests still counts 56 tests though, so going to find the one that does not belong to remove.

@lixuna

[agentpoyo]

rolling_update is listed in the 1.0-beta list of tests but it is not marked as a cert in the points.yml.

@lixuna

[lixuna]

completed, see https://github.com/cncf/cnf-certification/blob/main/docs/CNFCertification-1.1-beta.md