search

cnti-testcatalog / testsuite

📞📱☎️📡🌐 Cloud Native Telecom Initiative (CNTI) Test Catalog is a tool to check for and provide feedback on the use of K8s + cloud native best practices in networking applications and platforms
https://wiki.lfnetworking.org/display/LN/Test+Catalog
Apache License 2.0
169 stars 70 forks source link

[BUG] Problem with cluster-tools using TTY as input #2092

Closed rich-l closed 2 days ago

rich-l commented 3 days ago

cnf-testsuite version - v1.3.0 test platform - OpenShift 4.12

I noticed essential test specialized_init_system, single_process_type, zombie_handled, and sig_term_handled were all failing. It looks like there is an issue with running the cluster-tools with TTY as input. See logs below. Similar logs are seen in each of the essential test that failed (specialized_init_system, single_process_type, zombie_handled, sig_term_handled).

I, [2024-06-27 11:52:39 -07:00 #907] INFO -- cnf-testsuite: KubectlClient::Get.pods command: kubectl get pods --all-namespaces -o json I, [2024-06-27 11:52:42 -07:00 #907] INFO -- cnf-testsuite: KubectlClient.exec command: kubectl exec -n cnf-testsuite -ti cluster-tools-skf79 -- crictl inspect cri-o://41d0599dd7f923016a093b27773ec80bba2639b1eff95e4e392bc805aca95a93 I, [2024-06-27 11:52:45 -07:00 #907] INFO -- cnf-testsuite: KubectlClient.exec stderr: Unable to use a TTY - input is not a terminal or the right kind of file time="2024-06-27T18:52:45Z" level=fatal msg="failed to connect: failed to connect, make sure you are running as root and the runtime has been started: context deadline exceeded" command terminated with exit code 1 E, [2024-06-27 11:52:45 -07:00 #907] ERROR -- cnf-testsuite: container_id not found for: cri-o://41d0599dd7f923016a093b27773ec80bba2639b1eff95e4e392bc805aca95a93

martin-mat commented 3 days ago

This looks like cluster-tools pod not in "Ready" state. Please make sure successful execution of 

cnf-testuite cnf_setup cnf-config=....

(watch the warning message "The ClusterTools installation timed out. Please check the status of the cluster-tools pods.") and check status of cluster-tools pods

kubectl get pods -n cnf-testsuite

Note that when having slower network connection, it might get longer time for cluster-tools to get pulled from the registry and this is most frequently the reason of such problems.

Also note that this is being addressed by optimizing cluster-tools image size cnf-testsuite/cluster_tools#24

rich-l commented 2 days ago

The cluster-tools are "Ready" NAME READY STATUS RESTARTS AGE cluster-tools-575wh 1/1 Running 0 14m cluster-tools-cqttr 1/1 Running 0 14m cluster-tools-vmxnf 1/1 Running 0 14m

rich-l commented 2 days ago

Output from cnf-testsuite specialized_init_system -lINFO > test-run-info.log

test-run-info.log

martin-mat commented 2 days ago

Openshift uses cri-o container engine. This is not compatible with cnf-testsuite requirements, which states containerd: https://github.com/cnti-testcatalog/testsuite/blob/main/INSTALL.md#minimum-requirements

I reproduced the same behavior/error on coredns example when deploying minikube with cri-o container engine.

rich-l commented 2 days ago

I will create a Feature request to address this issue.