search

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
43 stars 34 forks source link

[FEA] Revisit/Update eventlogs of scala unit tests #1154

Open nartal1 opened 2 days ago

nartal1 commented 2 days ago

Is your feature request related to a problem? Please describe. Some of the unit tests in core/ test against static eventlogs which could be old/doesn't have updated information. Some of them were generated on 3.1.x which will be deprecated soon on the rapids-plugin. Also, we have some tests which has eventlog text in the tests. This is from a discussion https://github.com/NVIDIA/spark-rapids-tools/pull/1151#discussion_r1658974477

This is a tracking issue to revisit all the unit tests and maybe generate eventlogs on a newer Spark version so that the unit tests have better coverage.

amahussein commented 18 hours ago

This could be a part of our E-2-E testing revamp as we re-evaluate our pre-commit tests