Monitoring options

[MrF3lix]

As a product maintainer I want to be able to monitor the cluster, the status of each application and the amount of errors during execution.

Acceptance Criteria

• All applications should have endpoints for Prometheus to gather system / data flow metrics
• The metrics should be displayed on a Grafana Dashboard

Information

• Deploy new helm chart to k8s cluster
  • Prometheus
  • Grafana
• Add Prometheus client libraries to all application
  • prometheus-net
  • prom-client
  • postgres monitoring
• Configure alerts via teams/email/discord
• Add Health Check to API

Related

The following issues have been created as sub-issues:

• [x] #101
• [ ] #102

[S4nvers]

Possible metrics to gather

Editor

System metrics

• Exceptions => Site crashes (404, 500)
• Page views
• Visitors

API

System metrics

• Request count
• Request time
• Resource usage of executor
• Latency to db
• Exceptions

DataFlow metrics

• [x] Number of data flow requests as a list
• [x] Number of data flow requests by id
• [x] Number of edits
• [x] Number of created data flows
• [x] Number of deleted data flows

Linearization metrics

• [x] Number of linearization savings

User metrics

• [ ] Number of Users (Blocked by #90)
• [ ] Number of Active Users (Sign in in the last 7 days) (Blocked by #90)

Executor

System metrics

• [x] Resource usage of executor
• [x] Request duration
• [ ] Latency to other services
• [x] HTTP request types

Dataflow metrics

• [x] Number of dataflows that failed to complete
• [x] Number of dataflows that completed successfully

Execution metrics

• [x] Execution count (can be calculated using failed and successful dataflow)
• [x] Dataflow execution time
• [ ] Number of dataflows that hit execution time limit
• [ ] Number of dataflows that hit execution count limit

[MrF3lix]

The following issues have been created as sub-issues:

• 101

• 102

[MrF3lix]

@S4nvers The status page will be in a separate use story #103