Adding better configurations for prometheus metrics so metrics displa…

radanalyticsio / openshift-spark

72 stars 83 forks source link

Adding better configurations for prometheus metrics so metrics displa… #77

Closed zak-hassan closed 5 years ago

zak-hassan commented 5 years ago

Hi there,

We found that users would like to know how much resources did spark take per application run. To have a cleaner prometheus metrics, I've included a way to relabel prometheus metrics to display metrics based with a prefix.

All metrics for:

Spark master will have prefix: spark_master_****
Spark worker will have prefix: spark_worker_****
Spark master will have prefix: spark_driver_****

elmiko commented 5 years ago

hey @zmhassan , thanks for the pr!

this looks good on the surface, gonna give it a test drive today.

zak-hassan commented 5 years ago

Sounds good @elmiko 👍

elmiko commented 5 years ago

just for completeness, my metrics logs

metrics-master.txt metrics-worker.txt

crobby commented 5 years ago

Changes lgtm.

tmckayus commented 5 years ago

LGTM, looks like all the changes were generated for the different spark yamls afaict