Understanding the performance of Materialize is difficult. We have a set of queries support uses to understand performance effects, but running through the queries, putting their results together and forming a conclusion is hard because it requires specific knowledge. We propose to offer a self-service performance tool that presents all data we currently retain in a structured and digestible way, and on top of this adding more data that we do not collect by default.
Specification
TODO
Design
See #17858.
Documentation
No response
Required Issues (Must Haves)
[x] Determine the parent worker of a Timely worker
[x] Introduce a mapping from channels to operators
[ ] Dataflow graph visualization
[ ] Dependency visualization
[ ] Saving query metadata (X-dep on Surfaces?)
Discretionary Issues (Nice To Haves)
[ ] #15244
[ ] #17922
[ ] Logging of introspection data through environmentd or enabling persisted introspection sources.
Product Outcome
Understanding the performance of Materialize is difficult. We have a set of queries support uses to understand performance effects, but running through the queries, putting their results together and forming a conclusion is hard because it requires specific knowledge. We propose to offer a self-service performance tool that presents all data we currently retain in a structured and digestible way, and on top of this adding more data that we do not collect by default.
Specification
TODO
Design
See #17858.
Documentation
No response
Required Issues (Must Haves)
Discretionary Issues (Nice To Haves)
Blockers
No response