Migrate Spark 3 metrics to use PrometheusServlet

kubeflow / spark-operator

Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.

Apache License 2.0

2.77k stars 1.38k forks source link

Migrate Spark 3 metrics to use PrometheusServlet #1141

Open dszakallas opened 3 years ago

dszakallas commented 3 years ago

We could utilize Spark 3's prometheus support, instead of using the JMX exporter. This would get rid of the additional complexity and CPU overhead of running the exporter agent. However, I am almost 100% sure this is going to change the exposed set of metrics and their names. Spark's documentation is not to great on the PrometheusServlet usage, here's a better tutorial: https://dzlab.github.io/bigdata/2020/07/03/spark3-monitoring-1/.

arghya18 commented 3 years ago

Any update on this?

github-actions[bot] commented 3 days ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.