Assess available open solutions for metrics collection, roll-up and dashboarding.

[nilshoffmann]

In one of the recent WP5 Meetings, ELIXIR LU presented their Datacatalog solution to collect data and study metadata https://datacatalog.elixir-luxembourg.org/ With some extensions, this could serve as an aggregation point for Tool and Workflow metrics provider registration. However, for roll-up reporting and visualization, their data schema may not be suitable, depending on the expected traffic. de.NBI Cloud is using Elastic Search (FileBeat and MetricBeat) to record monitoring different OS and other metrics over time and query / visualize this via grafana.

• [ ] Discuss necessary schema changes and setup for providers in Datacatalog (maybe new name: Tool Metrics Catalog?)
• [ ] Test deployment and at least one test provider (potentially with simulated tool metrics at first) using the schema defined in #4
• [ ] Define a Grafana dashboard that can display a selection of metrics
• [ ] Survey existing solutions for tracing and metrics collection (e.g. OpenTelemetry) and suitable databases

[nilshoffmann]

For a first prototype, Grafana can also pull from Postgres and even REST APIs with additional plugins.