Avoid round trips from job-entities -> uuids -> job-entities in generating /list API endpoint.

twosigma / Cook

Fair job scheduler on Kubernetes and Mesos for batch workloads and Spark

Apache License 2.0

338 stars 63 forks source link

Avoid round trips from job-entities -> uuids -> job-entities in generating /list API endpoint. #790

Open scrosby opened 6 years ago

scrosby commented 6 years ago

In cook.mesos.api/list-jobs, we lookup jobs by entity, then map them to UUID. Then in https://github.com/twosigma/Cook/blob/master/scheduler/src/cook/mesos/api.clj#L2164, we call cook.mesos.api/fetch-job-map that maps the UUID's back into job-entities. Lets not do this round trip.

dposada commented 6 years ago

Is this a duplicate of issue #617 ?

DaoWen commented 6 years ago

I'm closing this as a duplicate of #617—but I reported #617, and I wholeheartedly agree that we should fix this!

scrosby commented 6 years ago

Its related to #617 in that I'm avoiding round-tripping to and from UUID. But its not a duplicate. Fixing this does not involve any changes to what goes into the liberator context --- the dataflow is via function arguments.