search

kernelci / kernelci-project

KernelCI Linux Foundation project documentation
10 stars 26 forks source link

Misleading test results in tast test: filemanager.ZipPerf #396

Open r-c-n opened 3 months ago

r-c-n commented 3 months ago

Example test result node: https://staging.kernelci.org:9000/viewer?node_id=666a62a8862fd68358578416 Log: https://people.collabora.com/~rcn/logs/kernelci_issues/log_filemanager_ZipPerf.txt

In this test group, all children test cases are marked as failed. However, only one of them ("unzip_500_small_files") seems to have explicitly failed, although this is not 100% clear from the log.

Is LAVA interpreting the log results correctly?

nuclearcat commented 3 months ago

It seems tast tests way/interpretation, as i see in LAVA job output, we have tast tests output: "2024-06-13T03:07:28.035455Z filemanager.ZipPerf [1;31m [ FAIL ] [0m500_small_files: Failed to get zip time histogram update: context deadline exceeded during a poll with timeout 10s; last error follows: histogram unchanged"

r-c-n commented 3 months ago

Yes, that's about that particular test case, but I can't find any results about the rest of the test cases, which are interpreted as failed.

nuclearcat commented 3 months ago

They are likely generated internally by tast tools and we decode them from json file generated by tast test.