Split JobStageAggTaskMetrics file into two different files

[parthosa]

Issue: #1017

This PR splits the job_+_stage_level_aggregated_task_metrics.csv generated by the Profiling tool into two separate files. We have fixed this in Qualification Tool (#1000).

Changes

• Create two separate case class JobAggTaskMetricsProfileResult and StageAggTaskMetricsProfileResult that extend a base trait (since all methods and arguments are same).
• Fix the AutoTuner code that checks the shuffle-skew
• Rename ID column to jobId and stageId correspondingly.
• Update heuristics view to use the new file and new column name.
• Update unit tests.

Note

• This PR will be merged in a staging branch
• A follow up PR on the staging branch will update estimation model to use the new file.
• After that we can merge the staging branch in dev.

Not Covered

• Add a field "attemptId" in stage_level_aggregated_task_metrics.csv
  • Tracking Issue: https://github.com/NVIDIA/spark-rapids-tools/issues/1048

[parthosa]

@amahussein Thanks Ahmed. We are targeting this PR to a staging branch.

After this, I will submit a follow up PR with changes only in the prediction code targeted to this staging branch.

This would make it easier for Lee/Eric to review the changes in prediction code only.

We can provide them the relevant changes in tools at an abstraction.

Once the follow up PR is merged on staging branch, we can then merge the staging branch to dev. Thus, we should be able to not fallback to legacy speedups.

[parthosa]

@amahussein The changes in prediction code are very minor. Hence I preferred making this change before.