scylladb / argus

Apache License 2.0
4 stars 11 forks source link

feature(graphs): Added graphing functionality for test results #432

Closed soyacz closed 2 months ago

soyacz commented 3 months ago

This commit introduces the ability to display charts from test results. Users can view these charts by clicking a button next to the runs list.

Each chart represents a single column value from the results table, with each row displayed as a series. For instance, a table with 4 columns and 3 rows will generate 4 charts, each with 3 series.

Given that some tests generate a large number of results, especially when combined with workload variations (e.g., in performance tests), the number of charts can be substantial. To manage this, users can filter visible charts using automatically generated filters. These filters are created by splitting the result table name by '-', where each segment becomes a filter. For example, 'read - Steady State - latencies' will generate filters for [read, Steady State, latencies]. Additionally, each column generates another filter, enabling further refinement of the visible charts. Filters are grouped by 'level,' determined by the order of table name segments. In the previous example, 'read' is level 1, 'Steady State' is level 2, and so on.

Filters update dynamically as users select them, adjusting to the currently visible charts for further refinement.

Users can click on a data point in a chart to view the corresponding run details, similar to selecting a run from the list. If the run is not currently visible, it will be fetched from the Argus backend. Clicking the same point again will scroll to the run's details.

To help observe trends when one value significantly deviates from others, the vertical axis scale is limited to display 50% of the results plus a margin. Results outside this range are 'clipped' and displayed at the edges of the axis. The actual value can be seen by hovering over the point.

Refs: https://github.com/scylladb/qa-tasks/issues/1244

soyacz commented 3 months ago

Demo:

https://github.com/user-attachments/assets/c34276be-fc26-4eee-9535-25833e4b30dd

also available in staging: https://stg.argus.komachi.sh/ (I migrated recent scylla-master/enterprise elasticity/perf with nemesis test results to Argus)

roydahan commented 3 months ago

Nice start! I wonder if we can set default filtering for the most relevant parts of each test. For example, the default filter for test_latency_read would be to show ONLY p99 read latency graphs.

soyacz commented 3 months ago

Nice start! I wonder if we can set default filtering for the most relevant parts of each test. For example, the default filter for test_latency_read would be to show ONLY p99 read latency graphs.

Anything can be done - it's a matter of complexity added to code.

I believe test_latency_read will not have that number of graphs, so filtering proper results would be one click away. Still things get complicated when one test uses different workloads so might require 2 clicks for proper filtering. Possibly could improve it by better result tables granularity and names (so filters are better matching our needs).

One simple and quick solution would be remembering filters locally in user's browser, so next time looking at graphs recent filter will be turned on.

soyacz commented 3 months ago

I think of removing hiding filters when is not applicable (e.g. selecting 'latencies' all filters related stalls are hidden) to not cause confusion. Still it would be one per level (so can select 'read' but when selecting 'write', 'read' will be deselected). Another change I think of is to allow select multiple filters on lowest level which will act as 'or' (so can see 'read p99' along with 'write p99'). Let me know WDYT, and I'll take a look when I'm back.

soyacz commented 2 months ago

ok, merging as a 'nice start' and to be refactored later with improvements ideas above.