search

kernelci / kernelci-pipeline

Modular pipeline based on the KernelCI API
GNU Lesser General Public License v2.1
6 stars 18 forks source link

Unusual test name and result of `kunit` skip test #651

Closed JenySadadia closed 2 weeks ago

JenySadadia commented 1 month ago

An investigation is required for the couple of issues I observed for the kunit tests.

I noticed the below node with a comment-like description in its name i.e. "example_mark_skipped_test # SKIP this test should be skipped":

$ curl https://staging.kernelci.org:9000/node/665964f3f35620b4ed11befa
{"id":"665964f3f35620b4ed11befa","kind":"test","name":"example_mark_skipped_test # SKIP this test should be skipped","path":["checkout","kunit-x86_64","exec","example","example_mark_skipped_test # SKIP this test should be skipped"],"group":"kunit-x86_64","parent":"665964f3f35620b4ed11bef7","state":"done","result":"fail","artifacts":null,"data":{"error_code":null,"error_msg":null,"test_source":null,"test_revision":null,"platform":"kubernetes","device":null,"runtime":"k8s-gke-eu-west4","job_id":null,"job_context":null,"regression":null,"kernel_revision":{"tree":"android","url":"https://android.googlesource.com/kernel/common","branch":"android14-5.15","commit":"d94e16579b8bdab145ea9f655c60b83ee0b2edee","describe":"android14-5.15.149_r00-121-gd94e16579b8b","version":{"version":5,"patchlevel":15,"sublevel":null,"extra":"-30003-gd94e16579b8b","name":null},"patchset":null},"arch":"x86_64","defconfig":null,"config_full":null,"compiler":null,"kernel_type":null},"debug":null,"jobfilter":null,"created":"2024-05-31T05:49:39.298000","updated":"2024-05-31T05:49:39.298000","timeout":"2024-05-31T11:49:39.298000","holdoff":null,"owner":"staging.kernelci.org","submitter":"9790c38bd29c1375fc96ed92b9b4bfab","user_groups":[]}

Also, its result should be skip and not fail.

I also checked other test nodes and there are previous tests with name example_mark_skipped_test which looks fine. But the result is not for these nodes.

$  curl 'https://staging.kernelci.org:9000/nodes?name=example_mark_skipped_test&limit=1'
{"items":[{"id":"65a1355ee98651d0fe81e47c","kind":"node","name":"example_mark_skipped_test","path":["checkout","kunit-x86_64","exec","example","example_mark_skipped_test"],"group":"kunit-x86_64","parent":"65a1355ee98651d0fe81e479","state":"done","result":null,"artifacts":null,"data":{"kernel_revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"70d201a40823acba23899342d62bc2644051ad2e","describe":"v6.7-6264-g70d201a40823","version":{"version":"6","patchlevel":"7","extra":"-6264-g70d201a40823"}}},"debug":null,"jobfilter":null,"created":"2024-01-12T12:49:34.002000","updated":"2024-01-12T12:49:34.002000","timeout":"2024-01-12T18:49:34.002000","holdoff":null,"owner":"admin","submitter":null,"user_groups":[]}],"total":1471,"limit":1,"offset":0}
JenySadadia commented 1 month ago

Note: This issue was present even before re-structuing the test hierarchy. All the nodes in the hierarchy is with kind=test.

JenySadadia commented 2 weeks ago

https://github.com/kernelci/kernelci-pipeline/pull/625 fixed the issue of incorrect test results. So, the only issue remaining now is the unusual test names. I'll open a new issue for it.

JenySadadia commented 2 weeks ago

Opened https://github.com/kernelci/kernelci-pipeline/issues/666