camsas / firmament

The Firmament cluster scheduling platform
Apache License 2.0
415 stars 79 forks source link

Does Firmament support fine-grained resource allocation? #75

Open gabrielecastellano opened 3 years ago

gabrielecastellano commented 3 years ago

Hello guys,

I opened this issue only to make sure I correctly understood that Firmament does NOT support fine-grained resource allocation.

In my understanding, each task always creates one unit of atomic flow. This is to ensure the MCMF formulation works fine (otherwise task flows could be wrongly split on multiple paths). Therefore, Firmament assumes that all the tasks are equal in terms of resource consumption. Correct? Let suppose we have three tasks: T1 requires 2 CPU T2 requires 1 CPU T3 requires 1 CPU And one machine M1 featuring 4 CPU. When modeling the problem, if I understood correctly, I have to decide how much CPU the unit of atomic flow stands for. To avoid T1 to be split, I should consider the unit = 2 CPU. Therefore the min-cost flow problem will feature Three source nodes each with 1 unit, as you said, and 1 Machine node with capacity = 2. As a result, the solver will not be able to schedule T1, T2, and T3 on M1. (only two of them fit the flow formulation).

This is equivalent to consider the capacity as "number of supported tasks", rather than a more "granular" cpu value. Is this understanding correct?

Thanks