search

Linaro / squad

Software Quality Dashboard
GNU General Public License v3.0
59 stars 41 forks source link

Presented columns dont sum to the total test states #390

Closed ghost closed 6 years ago

ghost commented 6 years ago

See https://pmwg.validation.linaro.org/results/

In the screenshot you see a test which appears to be missing a count of a failure state

2_wa-lxc-test shows 426 passes + 0 fails = 427 total, which is incorrect.

I would have expected another column for incomplete (or whatever happened)

2_wa-lxc-test showing 426 passes + 0 fails + 1 incomplete = 427 total

image

mwasilew commented 6 years ago

@mike-holmes-linaro this is LAVA screen. Is this the same in SQUAD?

amrohassaan commented 6 years ago

I pinged @codehelp to confirm.

ghost commented 6 years ago

@mike-holmes-linaro - check the 2_wa_lxc-test and you will find either a SKIP or UNKNOWN test case result. Compare with https://staging.validation.linaro.org/results/244686

0_server-unittests-stretch | 278 | 0 | 281 |

https://staging.validation.linaro.org/results/244686/0_server-unittests-stretch?sort=-result#table

test_panda_lxc_template_lava_scheduler_app_tests_test_uboot_templates_TestUbootTemplates_____skipped__lxc-info_not_installed_ | unittests | skip |
test_multinode_mixed_deploy_lava_scheduler_app_tests_test_pipeline_TestYamlMultinode_____skipped__rpcinfo_returns_non-zero_for_nfs_ | unittests | skip |
test_secondary_connection_lava_scheduler_app_tests_test_pipeline_TestYamlMultinode_____skipped__rpcinfo_returns_non-zero_for_nfs_ | unittests | skip |

The number of SKIP and UNKOWN test cases are available to Squad but this particular LAVA table has never shown figures for SKIP or UNKNOWN dating back through all the days of V1.

ghost commented 6 years ago

@codehelp thanks, does that mean you think it should be added ? As a casual user I'd say so, when viewing it you expect totals to add up. I'd also say that clearly showing anomalies like UNKNOWN is a primary use case for a UI.

If UI space is a premium, showing the total only, and having it expand if clicked, or hovered over would help, turning that total a different colour on UNKNOWN also helps the casual scanner perhaps.

Small things admittedly

ghost commented 6 years ago

@mike-holmes-linaro - not for me to say whether Squad should or should not do anything. I'm not a user of Squad.

mwasilew commented 6 years ago

@mike-holmes-linaro please report the lava issue here: https://git.lavasoftware.org/. I'm closing this ticket as it doesn't relate to squad.