Feature Request: Performance Summary View Widget

[soyacz]

Problem:

In Argus View, identifying failed test results from the current week is challenging due to the large number of tests and runs. Users must also navigate into test run details to view failed result tables, making it difficult to grasp failures at a glance. Additionally, important tested versions can be buried in the results.

Proposed Solution:

A widget will be developed to simplify the view of weekly test results, emphasizing failures and presenting failure details upfront, without requiring the user to open test run details. It will also provide better visibility into both passed and failed tests.

Widget Features:

1. Sectioned View by SUT Version:

   • The widget will be divided into sections based on the System Under Test (SUT) version, such as Scylla, Scylla Manager, or Scylla Driver.
   • Each section will represent a version tested during the current week and will display test run results segregated into passed and failed lists.

2. Highlighting Missed or Unstarted Tests:

   • Tests that were not executed within the last 7 days or failed to start (e.g., due to infrastructure issues like AWS capacity) and have no SUT version assigned will be highlighted at the top of the widget.
   • These tests will be treated as failures and listed separately, outside of any SUT version section.

3. Test Run Lists:

   • Each SUT Version Section will contain test runs segregated into two lists:
     • Failed List: Displays all failed test runs and their failed result tables.
     • Passed List: Displays all successfully passed test runs.
   • Each test run in both lists will be clickable, allowing users to navigate to the detailed test run page if needed.
   • Each test run entry will show:
     • The count of error events associated with that test run.
     • Failed result tables will be displayed (collapsed by default for passed tests, visible for failed tests). of the error count and the failed result tables (collapsed by default).

4. Run Investigation Status:

   • Any test runs marked as "IGNORED" in their investigation status will be excluded from the analysis to avoid clutter.
   • Test runs that have failed result tables or error events but are manually marked as passed will still be listed in the passed list, with an indication

5. Widget Configuration:

   • Users can configure the widget by:
     • Setting a custom widget name.
     • Selecting which tests to monitor from the available test groups.
   • Widgets can be edited at any time to add/remove tests or change the name.

6. Multiple Widgets:

   • Users can create multiple widgets, each with a custom name and a specific set of tests to monitor.
   • For example, users could create widgets like "Vnodes Based Tests Summary" or "Tablets Based Tests Summary," with each widget tracking a different group of tests.

Out of Scope:

• Looking back at results from previous weeks.
• Showing detailed error events beyond the error count.
• Configurable monitoring windows (default is 7 days).

refs: https://github.com/scylladb/qa-tasks/issues/1784

[soyacz]

@fruch @roydahan please let me know if I got it right and my proposed solution fits our needs.