camsas / firmament

The Firmament cluster scheduling platform
Apache License 2.0
412 stars 77 forks source link

Introduce regular updates of time-dependent costs in flow scheduler. #28

Closed ms705 closed 9 years ago

ms705 commented 9 years ago

Some costs in the flow network can change over time (e.g., any cost based on time since submission a task). In order to update the appropriate edges in the flow network, we must repeatedly invoke FlowGraph::AddOrUpdateJobNodes(). If we do this too often, we will generate a huge number of incremental changes. Instead, we define a configurable minimum time interval at which the costs will be updated.

Notw that this does not guarantee updates every n seconds, however: if no events happen in the cluster, the scheduler won't be invoked and no updates will be made. This is fine, since we assume that the optimality of the previous solution still holds. We may want to re-visit this assumption later and trigger regular scheduler runs.

ms705 commented 9 years ago

Code review in change 237756.