search

LucaCanali / sparkMeasure

This is the development repository for sparkMeasure, a tool and library designed for efficient analysis and troubleshooting of Apache Spark jobs. It focuses on easing the collection and examination of Spark metrics, making it a practical choice for both developers and data engineers.
Apache License 2.0
690 stars 144 forks source link

AttributeError: 'StageMetrics' object has no attribute 'stagemetrics' #56

Closed cometta closed 4 months ago

cometta commented 4 months ago

i get above error when run the TPC example 

results = tpcds.run_TPCDS()
LucaCanali commented 4 months ago

Hi,

The examples related to running sparkMeasure with TPCDS workload can be found at https://github.com/LucaCanali/Miscellaneous/tree/master/Performance_Testing/TPCDS_PySpark They seem to work fine for me. Can you please check and in case provide more details?

Best, Luca

cometta commented 4 months ago

need to download the jar, and put spark.jars to get it works. why can't it read the jar directly from sparkmeasure python library. the jar already exist on the python directory.