developmentseed / segment-anything-services

Running segment-anything image embedding, prompting, and mask generation as torchserve services
Apache License 2.0
92 stars 10 forks source link

Autoscale CPU & GPU ECS Tasks #20

Closed ingalls closed 1 year ago

ingalls commented 1 year ago

Context

Scaling ECS Tasks is currently manually managed by the user when they need compute resources. In order to save money from forgotten compute resources, this PR introduces a scale down behaviour that after 20 minutes of no request activity to a given service, the service will be scaled down by -1 Task per 20 minute period.

cc @rbavery