search

camsas / firmament

The Firmament cluster scheduling platform
Apache License 2.0
412 stars 77 forks source link

Is there roadmap about integrate this with Kubernetes? #42

Closed resouer closed 8 years ago

resouer commented 8 years ago

Just as title said, I did not find any roadmap doc to describe when/how to integrate this with Kubernetes project.

ms705 commented 8 years ago

Hello,

Our Kubernetes integration work is happening in the Project Poseidon repo, while this one contains the core Firmament scheduler code and our research cluster manager (the confusingly-named coordinator build target).

Discussion of the Kubernetes port should happen in the Poseidon repo, but to answer your question: our road map for this is that we'll start working more seriously on it in mid-late May. We've already made a number of changes that will make it easier to use Firmament as a library within other cluster managers (8f4c8c08e175e6f1eef7f6d0052e2e1697bf115a is a key change), but currently we're working towards a bunch of research deadlines right now that will keep us busy until May.

The primary contributors to this project so far are all academic researchers -- while the Kubernetes integration (and other integrations) are a goal for us, they're outside our day-to-day work and we can only work on this (at best) part time.

That said, we're certainly happy to assist anyone who wants to help getting this effort off the ground. What's in the Poseidon repo currently amounts to a proof of concept that talks to the Kubernetes API server and instantiates a Firmament scheduler. The next steps to getting things to work end-to-end would be to compile a single binary that

  1. queries the API server's pods and nodes endpoints;
  2. converts the resulting Pod specifications into Firmament data structures (JobDescriptor and TaskDescriptor protobufs for pods, and ResourceDescriptor for nodes) and inserts them into the appropriate collections passed to FlowScheduler's constructor;
  3. replaces Firmament's LocalExecutor with a KubernetesExecutor that posts to the bindings API endpoint.

If all goes well, I suspect we'll have something working some time in June/July this year.

Closing this as further discussion should take place in the Poseidon repo.