search

coiled / feedback

A place to provide Coiled feedback
14 stars 3 forks source link

Add some flexibility around worker core limits #46

Closed mrocklin closed 4 years ago

mrocklin commented 4 years ago

If you have a worker limit of 100 you might think that you could scale to 25 4-core machines. Actually, this isn't the case, because the scheduler is there, taking up one core.

I recommend that we add a 10% buffer to limits to allow for a little bit of flexibility.

cc @TomAugspurger @marcosmoyano

necaris commented 4 years ago

@mrocklin right now we actually have some code to ensure the scheduler cores are included in the total count against which the limit is checked -- we could just remove it, if the goal is explicitly to limit worker cores.

mrocklin commented 4 years ago

I think that we should include scheduler cores generally. (we open ourselves up to people launching a lot of schedulers otherwise). I'm in favor of fudging by 10%.

On Mon, Aug 17, 2020 at 10:23 AM Rami Chowdhury notifications@github.com wrote:

@mrocklin https://github.com/mrocklin right now we actually have some code to ensure the scheduler cores are included in the total count against which the limit is checked -- we could just remove it, if the goal is explicitly to limit worker cores.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/coiled/coiled-issues/issues/46#issuecomment-675008308, or unsubscribe https://github.com/notifications/unsubscribe-auth/AACKZTHHX3CVLSN7I26ZCF3SBFRSXANCNFSM4P7YOMZA .