Closed pisarenko-net closed 1 year ago
Disclaimer: please review the metrics interactions carefully. I am not sure if anything else is required (like an explicit emit, or rate limiting). If there's a good primer on metrics please send me a link.
Thank you for the review Chris, I'm going to fix merge conflicts and incorporate feedback.
PR implements new metrics for packing efficiency. Packing is now run with 2 functions: the configured one and "pack tightly". The second round of packing is only used for reporting in metrics.
This depends on a new (unreleased) version of k8s-spark-scheduler-lib so this can't be merged until a new lib is released and dependencies are updated.