search

Netflix / Fenzo

Extensible Scheduler for Mesos Frameworks
700 stars 116 forks source link

Support v1 Protos (or provide utility to convert) #146

Closed zonybob closed 7 years ago

zonybob commented 7 years ago

I started porting our Fenzo-based scheduler over to the Mesos HTTP API using mesos-rxjava. One issue I ran into is documented in the mesos-rxjava project (https://github.com/mesosphere/mesos-rxjava/issues/74). To summarize, the HTTP API now hands me an org.apache.mesos.v1.Protos.Offer, but Fenzo only accepts org.apache.mesos.Protos.Offer. I have a few ideas for how to convert to/from these Offers, but they feel risky. I was wondering if (a) Fenzo plans to support v1.Protos in the future? ... and (b) if you had any idea how to cleanly convert from v1.Protos to legacy Protos in the meantime?

spodila commented 7 years ago

Conversion may work for the short term, and that might be the easiest way for you to proceed, I think.

An alternative to supporting v1.Protos.Offer (and any future versions) in Fenzo is a thought to abstract the Offer object in Fenzo and then have a set of Provider implementations for them. So, a v0 user would use the v0 provider, you'd use a v1 provider, etc. However, this could potentially break backwards compatibility due to change in Fenzo's VirtualMachineLease.getOffer() signature. We will have to consider this further.

Additional thoughts welcome.

zonybob commented 7 years ago

That was the general idea I was hoping for. It seems like most Fenzo interfaces abstract away Mesos-specific concepts and use Fenzo data models. Perhaps only Protos.Attribute and Protos.Offer exist as hard ties to the Mesos library. It might be nice to see those get abstracted in the future as well, in favor of adapters/providers as you mention. In the meantime, I'm posting the question to the Mesos community of how to safely convert v1 models to the older models. Thanks for the feedback!

zonybob commented 7 years ago

FWIW, on Mesos Slack, someone pointed me to this link. The class notes that Mesos uses the same conversion logic internally, so that was good enough for me. It is basically serialize, deserialize... but using the builder.merge.

https://github.com/apache/aurora/blob/c85bffdd6f68312261697eee868d57069adda434/src/main/java/org/apache/aurora/scheduler/mesos/ProtosConversion.java

spodila commented 7 years ago

Nice, thanks for the update.

spodila commented 7 years ago

I am thinking that the workaround solves this issue and therefore I'm going to close this. Feel free to reopen this or a new one if needed. Thank you.