[ML] Discuss Potential Enhancement for `ml.allocated_processors_scale` to Support Expansion

[Rassyan]

Description

I'd like to discuss a potential enhancement to the ml.allocated_processors_scale setting introduced in PR #98296. The current design appears to focus on scaling down processor counts to account for hyper-threading, which is a valuable feature.

However, in scenarios where nodes have excess capacity, it could be beneficial to allow for scaling up the processor count to enable more model allocations. This could provide users with additional flexibility in resource-rich environments.

I propose we consider the following enhancements:

1. Given that ml.allocated_processors_double accepts floating-point values, aligning ml.allocated_processors_scale to also accept floating-point values would enhance precision.
2. Enable ml.allocated_processors_scale to support values less than 1 to effectively increase the processor count for model planning.
3. Add documentation to clearly articulate the setting's impact on model allocations and threads.

I believe these enhancements could make the setting more versatile and user-friendly. I'm open to further discussion and willing to contribute to the implementation of these improvements.

Thank you for considering this proposal.

[elasticsearchmachine]

Pinging @elastic/ml-core (Team:ML)