[FEA] Add multitenant support for qualification user tools for more accurate cost calculations

[mattahrens]

Use case: a customer is running Spark CPU applications on a cluster where other logs are running. The qualification tool calculates CPU cluster cost for a job assuming the CPU application is using the entire cluster. We can add a mode via config that will adjust CPU cluster cost by the executor usage for the application.

Requirements:

1. Add a new option in the user tools qualification command multitenant that will support new CPU cluster cost calculation.
2. When multitenant is set, the CPU cluster cost calculation should be the given total cluster based on worker nodes divided by the number of executors that show up in the event log. Specific formula (draft): multitenant CPU cost = total cost / (cluster cores/scale_factor / active executors). TBD on what to use for scale_factor.
3. Estimated GPU cost should be updated based on multitenant CPU cost.

Additional idea is to run NDS throughput benchmark for CPU/GPU clusters to generate adjusted speedup factors for concurrent jobs.

[amahussein]

Cost savings are disabled in qualification