mesos / chronos

Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules
http://mesos.github.io/chronos/
Apache License 2.0
4.38k stars 529 forks source link

Support for constraints in chronos #256

Open mattomatic opened 9 years ago

mattomatic commented 9 years ago

Is it possible to add support to chronos for constraints, such as those that marathon already supports (https://mesosphere.github.io/marathon/docs/constraints.html)?

For example, it would be very useful to be able to schedule jobs to be run in a particular data center.

Also, is there an existing workaround for this?

brndnmtthws commented 9 years ago

It's feasible, but not currently on any sort of roadmap. PRs welcome.

elingg commented 9 years ago

Adding constraints would be an important feature for Chronos. Patches from the community would be excellent if we aren't able to add this in our immediate roadmap.

jwerak commented 9 years ago

Having marathon-like constrains feature would be super cool - https://mesosphere.github.io/marathon/docs/constraints.html

kutchar commented 9 years ago

+1

mjbroekman commented 9 years ago

+1 ... having the ability to specify parts of your cluster to run jobs on would great...

synergyhue commented 9 years ago

+1

ghost commented 9 years ago

+1

siddharth178 commented 9 years ago

+1

freshmatrix commented 9 years ago

+1

SEJeff commented 9 years ago

Hey gents, instead of a pointless +1, just click the Subscribe button on the top right. More and more +1s are not very useful and just spam the developers unnecessarily.

ssk2 commented 9 years ago

@SEJeff, the +1s can be helpful for prioritisation. Ideally we'd have a less spammy system or the ability to vote built into GH issues, of course.

SEJeff commented 9 years ago

That is what subscribe is for :)

I comaintain a few largish open source projects (> 500 unique contributors each), salt and graphite. It works pretty well.

gyllen commented 9 years ago

@brndnmtthws How hard would this be to accomplish, do you have any idea. It is most certainly an important feature.

kapil-malik commented 9 years ago

This works pretty well now for us in latest version. Thanks. Curious why this issue is still open now?

potto007 commented 9 years ago

Isn't it still restricted to just the EQUALS constraint at this point? I think the intent is to bring it to parity with Marathon wrt constraints.

kapil-malik commented 9 years ago

Oh yes, agreed.EQUALS was serving our purpose, didn't realize the full story. Thanks.

goodwillcoding commented 8 years ago

Is there a way to set constraints using the gui?

ssk2 commented 8 years ago

@goodwillcoding - not yet but we're working on revamping the GUI!

tony-kerz commented 8 years ago

is there any support for running a job on every slave in a cluster?

brndnmtthws commented 8 years ago

Not yet. On Sep 4, 2015 1:27 PM, "tony kerz" notifications@github.com wrote:

is there any support for running a job on every slave in a cluster?

— Reply to this email directly or view it on GitHub https://github.com/mesos/chronos/issues/256#issuecomment-137711020.

tony-kerz commented 8 years ago

is something like below supposed to work?:

  "constraints": [
    ["hostname", "EQUALS", "ip-10-0-0-95.ec2.internal"]
  ],
solarkennedy commented 8 years ago

Yes. EQUALS is one of the supported constraints per the docs: https://mesos.github.io/chronos/docs/api.html#constraints

tony-kerz commented 8 years ago

thx @solarkennedy, i saw that, but it wasn't clear to me if hostname was an attribute that was auto-magically made available to chronos for use with EQUAL/LIKE constraints or if those would only work on attributes that i manually punched into a slave via something like --attributes (which i found somewhere in the mesos doc), or MESOS_ATTRIBUTES (which i see in the cloud-formation file provided by mesosphere).

looks like marathon is pretty good about setting up hostname for use with constraints, but i saw no such info in the chronos doc.

solarkennedy commented 8 years ago

That is true, the docs do assume that the attribute used by the constraint is declared. As far as I know hostname is not a magic attribute that you get for free.

Can you make a PR to clarify that and cross-link to the mesos docs on how to add attributes? If not I could do it.

tony-kerz commented 8 years ago

@solarkennedy i can take a whack at clarifying the docs.

more importantly tho, i think it would be great if hostname was made available by default for constraints as it is in marathon. do you think this would be a valuable feature? if so, do you think it is (or should be) implied by this particular issue, or do you think it should be spiked out as a sep enhancement request?

solarkennedy commented 8 years ago

I'm not sure. I was not aware that that was a "free" constraint that you could use from marathon, even if you didn't set that as a mesos attribute. (I assumed the constraint language only operated on mesos attributes)

tony-kerz commented 8 years ago

check the last sentence of the first paragraph at this link: https://mesosphere.github.io/marathon/docs/constraints.html

The field can be the slave hostname or any Mesos slave attribute.

i've constrained to hostname successfully in marathon as well without doing anything special.

solarkennedy commented 8 years ago

Ah! Today I learned. @tony-kerz for this specific feature I recommend making a separate issue/pr.

If you ask me this main issue should be closed, as there is some constraint functionality. Any new feature requests or doc updates and be new things.

anhcuong commented 8 years ago

Please support the hostname constraint in the future because I guess many people are using Marathon along with Chronos on top of Mesos. And Marathon already supported hostname constraint

tony-kerz commented 8 years ago

fyi @anhcuong i have had #563 open for a while to provide that feature.

alex-hoang-piksel commented 6 years ago

Is there any news on adding the constraints field to the UI? Currently when I set a constraint on a job, if the job is then edited and saved via the UI, the constraints field is emptied.

anhcuong commented 6 years ago

@alex-hoang-piksel you may want to check out Airflow (https://airflow.incubator.apache.org/). They provided many more features with a rich UI. And yeah, the repository is more active than Chronos.