search

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
49 stars 36 forks source link

[FEA] Extend max-event-log-size log filtering #1276

Open tgravescs opened 1 month ago

tgravescs commented 1 month ago

Is your feature request related to a problem? Please describe. https://github.com/NVIDIA/spark-rapids-tools/pull/1275 adds an option to filter event logs by a maximum size. This doesn't currently work for rolled event logs. We should add that functionality. We should also think about adding it to the profiling tool as an option.

We may also want to think about having a range of sizes.. like a min and a max.

tgravescs commented 1 month ago

https://github.com/NVIDIA/spark-rapids-tools/pull/1291 adds in the minimum size filter option