search

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
54 stars 37 forks source link

[BUG] spark_rapids qualification outputs 'Invalid Jar Argument' when tools jar is passed and other minor misleading messages #977

Closed tgravescs closed 5 months ago

tgravescs commented 6 months ago

Describe the bug I called the tool with:

spark_rapids qualification \
  --verbose \
  --estimation_model xgboost \
  --tools_jar ./rapids-4-spark-tools_2.12-24.02.4-SNAPSHOT.jar \
  --eventlogs ./core/src/test/resources/spark-events-qualification/dsAndDf_eventlog.zstd

The output had a couple of lines that could be confusing to the user:

2024-04-30 09:01:08,899 INFO spark_rapids_tools.argparser: ...applying argument case: Invalid Jar Argument

We should remove or clarify this info message.

Other ones should be clarified like: 2024-04-30 09:01:08,900 INFO spark_rapids_tools.argparser: ...applying argument case: Define Platform based on Eventlogs prefix

Ideally tells us what the platform was set to.

2024-04-30 09:01:08,927 INFO rapids.tools.qualification: RAPIDS accelerator jar is downloaded to work_dir /home/tgraves/workspace/spark-rapids-tools2/qual_20240430140108_00d879F8/work_dir/rapids-4-spark-tools_2.12-24.02.4-SNAPSHOT.jar

'RAPIDS accelerator jar' should say RAPIDS accelerator tools jar

Steps/Code to reproduce bug Please provide a list of steps or a code sample to reproduce the issue. Avoid posting private or sensitive data.

Expected behavior A clear and concise description of what you expected to happen.

Environment details (please complete the following information)

Additional context Add any other context about the problem here.

amahussein commented 6 months ago

More like an incorrect log message. the Jar used in the runtime was the correct one specified by the CLI argument.