search

LucaCanali / sparkMeasure

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of Spark metrics, making it a practical choice for both developers and data engineers.
Apache License 2.0
690 stars 144 forks source link

Send to Prometheus not available from Python #28

Closed nadenf closed 4 years ago

nadenf commented 4 years ago

It is not possible to call:

  def sendReportPrometheus(serverIPnPort: String,
                 metricsJob: String,
                 labelName: String = sparkSession.sparkContext.appName,
                 labelValue: String = sparkSession.sparkContext.applicationId): Unit

From Python.

LucaCanali commented 4 years ago

It should be possible to call from Python. I think you can experiment with using something like this example:

stagemetrics = StageMetrics(spark)
...
stagemetrics.stagemetrics.sendReportPrometheus(serverIPnPort, metricsJob, labelName,labelValue)