search

NVIDIA / spark-rapids-tools

User tools for Spark RAPIDS
Apache License 2.0
54 stars 37 forks source link

[BUG] spark.task.resource.gpu.amount improvements #1401

Open kuhushukla opened 2 weeks ago

kuhushukla commented 2 weeks ago

Describe the bug Once we fix https://github.com/NVIDIA/spark-rapids-tools/issues/1067 , we should make sure that spark.task.resource.gpu.amount is revised as well.

amahussein commented 2 weeks ago

There are existing issues for AutoTuner. It is hard to know if those new ones are any different from previous issues. Searching on existing issues, there is a couple of ones related to gpu.amount issue-1274:FEA AutoTuner on GPU event log should verify spark.task.resource.gpu.amount setting