search

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
43 stars 34 forks source link

Handle different exception thrown by incomplete eventlogs #1124

Closed amahussein closed 2 weeks ago

amahussein commented 2 weeks ago

Signed-off-by: Ahmed Hussein (amahussein) a@ahussein.me

Fixes #1122

In some cases, Spark throws JsonPraseEsception with incomplete eventlogs. This change handles this case by checking the Ex message. If it is Incomplete eventlog, the tools does not consider the eventlog as a failure. Instead, it considers it as incomplete and guesstimate the AppEndTime.