flux-framework / flux-k8s

Project to manage Flux tasks needed to standardize kubernetes HPC scheduling interfaces
Apache License 2.0
21 stars 10 forks source link

Initial Assessment for Scheduler Plugin #1

Open cmisale opened 3 years ago

cmisale commented 3 years ago

As discussed during our meeting on Feb 3rd, we have two main options to build the Fluxion Scheduler Plugin

  1. Extension Points (Scheduling Framework)
  2. Out-of-tree Scheduler Plugin

We would need to start a discussion about pros/cons of the two options, so that we can take a decision ASAP. Initial metrics we can consider are:

Regardless of the option we go for, there are certain tasks we have to address that we identified during our discussion:

milroy commented 3 years ago

I'm going to create issues for each of the tasks you define here so we can track them separately.

milroy commented 3 years ago

I've opened several derived issues based on this one. Have I missed anything?

@cmisale: let's limit this issue to a discussion of the pros and cons of making use of the Scheduling Framework versus an out-of-tree scheduler plugin approach. Does that sound reasonable?

swatisehgal commented 3 years ago

Please refer to the link here which describes how the scheduler plugin allows us to write a custom scheduler without forking the scheduler's code. The scheduler-plugins repo here: https://github.com/kubernetes-sigs/scheduler-plugins/blob/master/cmd/scheduler/main.go is an example of a custom scheduler. So essentially in your above issue description where you mention building Fluxion Scheduler Plugin either with the scheduler framework or as an out of tree scheduler plugin please keep in mind that you could use both and develop a customized out-of-tree scheduler plugin that uses the scheduler framework!

dongahn commented 3 years ago

Oh this is a great resource, @swatisehgal! Thank you for chiming in.