search

dagster-io / dagster

An orchestration platform for the development, production, and observation of data assets.
https://dagster.io
Apache License 2.0
11.06k stars 1.38k forks source link

Show sum/mean/median of an asset plot for all partitions #21000

Open ion-elgreco opened 4 months ago

ion-elgreco commented 4 months ago

What's the use case?

We can see the stats per asset partition but I also would like to see the overall stats, would be useful if we can see for each plot:

image

The following aggregations: sum/mean/median on an asset metadata

Ideas of implementation

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

bengotow commented 4 months ago

Dug into this a bit, I think it's a bit tricky because we pull this data from the event logs, and not from a proper metrics system (eg: Dagster Plus Insights), so any average / sum would only be over the visible data (last 20 or 30 data points)

That said, I think we could add a line beneath each chart ala:

"Last 20 Points": "Avg=102, Median=49, Sum=123912

bengotow commented 4 months ago

Tracking this in Linear here: https://linear.app/dagster-labs/issue/FE-319/show-summeanmedian-of-an-asset-plot-for-all-partitions