search

kernelci / kernelci-core

Core KernelCI tools
https://kernelci.org
GNU Lesser General Public License v2.1
104 stars 97 forks source link

LAVA jobs: missing tast tests #2588

Open JenySadadia opened 3 months ago

JenySadadia commented 3 months ago

I have observed an issue with some LAVA tast jobs where LAVA job state is Completed and tast test suite result is pass. However, there are no tast sub-tests that could run other than setup tests such as os-release and tast-tarball.

For example https://lava.collabora.dev/scheduler/job/14549057

The logs look like below:

+ ./tast_parser.py --results
Unsupported data type                     'list_of_scalar_values', skipping                         Apps.ClamshellLauncher.AnimationSmoothness.Close.average
Unsupported data type                     'list_of_scalar_values', skipping                         Apps.ClamshellLauncher.AnimationSmoothness.OpenAppsPage.average
<LAVA_SIGNAL_TESTSET START ui.DesktopControl>
Received signal: <TESTSET> START ui.DesktopControl
Starting test_set ui.DesktopControl
<LAVA_SIGNAL_TESTSET STOP>
Received signal: <TESTSET> STOP
Closing test_set ui.DesktopControl
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.InteractiveWindowResize.TimeToPresent.average
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.Overview.AnimationSmoothness.Enter.ClamshellMode.average
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.Overview.AnimationSmoothness.Exit.ClamshellMode.average
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.Window.AnimationSmoothness.CrossFade.average
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.Window.AnimationSmoothness.CrossFade.DragMaximize.average
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.Window.AnimationSmoothness.CrossFade.DragUnmaximize.average
Unsupported data type                     'list_of_scalar_values', skipping                         Ash.WindowCycleView.AnimationSmoothness.Show.average
<LAVA_SIGNAL_TESTSET START ui.WindowControl>

If the tests are being skipped, it should be reported with skip status or something else in the callback data but no such information is present.

a-wai commented 3 months ago

The tests are actually executed, the bug lies in our results parser which should report something instead of completely ignoring those.

I'll have a look ASAP.