apache / gravitino-playground

A playground to experience Gravitino
Apache License 2.0
31 stars 31 forks source link

[Enhancement] Integrate Prometheus and Grafana for monitoring in playground #96

Closed orenccl closed 2 weeks ago

orenccl commented 2 weeks ago

Closes #94

  1. Add monitoring information to the README
  2. Add Prometheus and Grafana settings to docker-compose
  3. Add a basic Prometheus configuration file
  4. Add a basic Grafana configuration, provision data source to Prometheus, and provision a dashboard to Gravitino Playground
  5. Add port checks in playground.sh

Fixes


Note: Please ignore the high JVM Non-Heap Memory Usage and JVM Direct Memory Usage caused by my WSL2 environment.

image

xunliu commented 2 weeks ago

@orenccl Very coooool !!!! Thank you for your contribution.

FANNG1 commented 2 weeks ago

cool

orenccl commented 2 weeks ago

At the first commit, gravitino-playground.json is a bit large(109KB). This file is a preset for the Gravitino dashboard. In the second commit, I reduced this file by minifying it, bringing the size down to 53KB (less than 50% of the original size).

Why this file is so large

  1. I included almost every metric provided by Gravitino so others don't have to create panels themselves.
  2. The preset dashboard provided on the official website is even 378KB. [Node Exporter Full with Node Name | Grafana Labs]

Therefore, I think 53KB is reasonable given the level of detail in the dashboard.

Below is current look:

image (1) image (4) image (5) image (6) image (7) image (8)

Hope 53KB is fine. If not, I can reduce it further by removing rarely used metrics, like this.

image (2)

Alternatively, I can split it into three different dashboards; each one will be smaller, but the total size will remain the same.

image (3)

orenccl commented 2 weeks ago

@xunliu I’d love to hear your perspective.

orenccl commented 2 weeks ago

Here’s an example of minified JSON for those who may not be familiar with it.

95b01da0-fab9-4fa3-aade-eb212a73a4d1-jsonformatter Credit: https://devutils.com/guide/json-formattervalidator-7/

orenccl commented 2 weeks ago

Hi @xunliu

The home path has been configured to go directly to the Gravitino dashboard.