numaproj-labs / numaflow-perfman

Performance testing framework for the Numaflow platform
Apache License 2.0
0 stars 1 forks source link

Improve the reporting experience #9

Open KeranYang opened 1 month ago

KeranYang commented 1 month ago

Problem Statement

There are a couple of issues with the report command:

  1. The user has to manually login to Grafana first to be able to open the snapshot url.
  2. The report only shows metrics for one vertex, to switch to a different vertex, the user has to go to Grafana to interact with the dashboard. It's not bad because the user should feel comfortable playing around with the dashboard.

Acceptance Criteria

Define a better experience and implement it.

Solutions to consider

  1. We could instead of generating dashboard snapshot url, write to a reporting file.
  2. Look into Grafana Reporting
ayildirim21 commented 4 weeks ago

Add functionality for user to also define a period of time, such as -t 600. With this flag we should collect data for the defined period of time and generate a report for it, so that users have a tangible basis for comparison for other generated reports.

KeranYang commented 4 weeks ago