search

huawei-cloudnative / firmament

The Firmament cluster scheduling platform
http://www.firmament.io
Apache License 2.0
19 stars 9 forks source link

Is there any docs for CPU-MEM cost model? I wonder the advantages & disadvantanges. #40

Closed cy-zheng closed 5 years ago

deepak-vij commented 5 years ago

See here. https://kubernetes.io/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler/

cy-zheng commented 5 years ago

One more question: Is there any plan to achieve data locality cost model, like Quincy?

deepak-vij commented 5 years ago

Not at this time. We are currently working on the complex Gang Scheduling, including proportions, fairness etc. You are welcome to work on it, if you would like.

cy-zheng commented 5 years ago

OK, thanks!

chauncey-77 commented 3 years ago

See here. https://kubernetes.io/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler/

The link is broken now.

deepak-vij commented 3 years ago

@anvithks

anvithks commented 3 years ago

See here. https://kubernetes.io/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler/

The link is broken now.

@deepak-vij The docs are based on Kubernetes version.

The documentation is available in the K8S v15 docs. Here you go https://v1-15.docs.kubernetes.io/docs/concepts/extend-kubernetes/poseidon-firmament-alternate-scheduler/

deepak-vij commented 3 years ago

@anvithks , all the links on the Poseidon blog are broken now for some reason. Someone pointed it out. Can we fix it. thanks.

deepak-vij commented 3 years ago

@anvithks, see the following link to the blog.

https://kubernetes.io/blog/2019/02/06/poseidon-firmament-scheduler-flow-network-graph-based-scheduler/

anvithks commented 3 years ago

@anvithks, see the following link to the blog.

https://kubernetes.io/blog/2019/02/06/poseidon-firmament-scheduler-flow-network-graph-based-scheduler/

I will see if I can update these when I have some free time.

The links just need to be updated to the v1. 15 docs linked above.

deepak-vij commented 3 years ago

@anvithks, thanks. Let me know when done so that I can let the community member know who wants to look at the docs. Thanks.

anvithks commented 3 years ago

@anvithks, thanks. Let me know when done so that I can let the community member know who wants to look at the docs. Thanks.

The docs are available here now. No need to wait.

deepak-vij commented 3 years ago

@chauncey-77, you can see the docs as @anvithks mentioned.

@chauncey-77, I will also publish the Google Doc for the CPU/Mem design on github.

chauncey-77 commented 3 years ago

It's very kind of you, thanks again!

chauncey-77 commented 3 years ago

@chauncey-77, you can see the docs as @anvithks mentioned.

@chauncey-77, I will also publish the Google Doc for the CPU/Mem design on github.

Hi, has the Coogle Doc for the CPU/Mem design been published on github? If it has, please share the link, thanks!

deepak-vij commented 3 years ago

Sorry, I have been quite busy and ongoing holidays stuff. I am still trying to dig out information we put together at the time. Have not been able to find it yet. I will publish something when ready. In the meanwhile, code is quite self explanatory w.r.t. multi-dimensional CPU/Mem based cost model.

chauncey-77 commented 3 years ago

Sorry, I have been quite busy and ongoing holidays stuff. I am still trying to dig out information we put together at the time. Have not been able to find it yet. I will publish something when ready. In the meanwhile, code is quite self explanatory w.r.t. multi-dimensional CPU/Mem based cost model.

Yes I am reading the function in cpu_cost_model.h/cc, and I have figured out most of them. But at the same time I also have many questions. For example, I still can't understand the construction process of flow graph, i.e., how to generate the flow graph according to the CPU/Mem scheduling strategy. The cpu_cost_model involves some types of graph nodes, such as EquivClass, task, UnSceduledAgg, resource and sink, but I don’t figure out how these nodes form the flow graph. When comparing this graph construction function and those mentioned in the Firmament paper, I think they are different. So is there an example to understand the flow graph construction process like the Firmament paper do?

deepak-vij commented 3 years ago

Yes, graph construction is different as we are dealing with complex multi-dimensional cost model. Firmament paper does not get into that. Give me some time, I will provide details.

chauncey-77 commented 3 years ago

Yes, graph construction is different as we are dealing with complex multi-dimensional cost model. Firmament paper does not get into that. Give me some time, I will provide details.

Oh this is awesome, I am very much looking forward to it.