search

coiled / feedback

A place to provide Coiled feedback
14 stars 3 forks source link

Accelerate Cluster startup on GCP #115

Closed mrocklin closed 2 years ago

mrocklin commented 3 years ago

Creating a cluster on GCP takes around 3 minutes. I imagine that this is going to be a slow-burning issue, but we'll probably want to look into accelerating this in the future.

In [9]: time cluster = coiled.Cluster(software="play", n_workers=5)
Creating Cluster. This takes about a minute ...Checking environment images
Valid environment image found
CPU times: user 6.08 s, sys: 851 ms, total: 6.93 s
Wall time: 3min 10s
necaris commented 3 years ago

@mrocklin does this happen every time you create a cluster, or just the first time? @selshowk mentioned that the way things are currently built, the first time will be slower.

mrocklin commented 3 years ago

This was not the first time I created a cluster

ntabris commented 2 years ago

GCP cluster start times (in the stress test we're running) are now around 2.25-2.5 minutes.

shughes-uk commented 2 years ago

Worth clarifying that our instance boot times are now about as fast as they will be, and the biggest gains will be had in future by figuring out how to pull/extract software environments faster