NVIDIA / spark-rapids-jni

RAPIDS Accelerator JNI For Apache Spark
Apache License 2.0
32 stars 60 forks source link

profile converter should have best effort mode to deal with corrupted/incomplete profiles #2174

Open jlowe opened 2 weeks ago

jlowe commented 2 weeks ago

Is your feature request related to a problem? Please describe. Sometimes profiles generated by the self-profiler are incomplete due to a violent shutdown of the executor. This leaves the compressed profile in a truncated state that cannot be decompressed successfully, and thus no profile data can be collected.

Describe the solution you'd like The profile converter should deal with compressed files directly, setting up a decompression stream and reading records from that stream. There should be a flag that can be passed that will read all of the complete records that it can before it encounters a decompression error and then work with the profile it has thus far to salvage what it can from the incomplete file