search

jitsi / jitsi-autoscaler

Jitsi Autoscaler microservice
Apache License 2.0
30 stars 18 forks source link

Do you support scale down? #157

Open NikitaSTets opened 4 months ago

NikitaSTets commented 4 months ago

As I see in the code you just logs scale down, and add audit events but no more

aaronkvanmeerten commented 4 months ago

The current code relies on the sidecar component to perform the final shutdown. This is because we were relying on the logic on the instance itself to know when it was ready to be shut down. I am planning functionality for serverside shut down sometime this year.

NikitaSTets commented 4 months ago

@aaronkvanmeerten Thanks for quick response, So just to be 100% sure, it means that now if we use digital ocean it will create droplets for scaling up, but when we need to scale down it just stop JVB, but droplet will be running, but you are going to implement mechanism to remove droplets, am I right?

aaronkvanmeerten commented 4 months ago

Actually, the digital ocean support was added via community, so I haven't been actively maintaining it. So I'd need help with that part.

The new feature I'm hoping to roll out is the shutdown hook that allows the sidecar to report back to the autoscaler to report final shutdown state.

Right now the instances are intended to just shut themselves down (via shell scripts triggered by the sidecar) without checking in to the server a final time. Each instance needs permissions to shut itself down, and is expected to run the appropriate shell command for the cloud provider.

Here's an example of what we do in oracle cloud: https://github.com/jitsi/infra-configuration/blob/d024c5fd380e8ed13e93e2512425ec87c8158272/ansible/roles/autoscaler-sidecar/files/terminate_instance_oracle.sh#L16

Once the final shutdown hook feature exists, we'd be able to trigger optional server-side termination. So I hope to also implement that at least in Oracle Cloud and Nomad.

In doing so I think it should be fairly easy to make it work for Digital Ocean as well, but I'd just be guessing and would need a user or developer to help test anything around that cloud.

All that to say, planned features are always subject to delay and change so take the above as my current snapshot of where I hope to arrive, rather than a published schedule of deliverables :)

NikitaSTets commented 4 months ago

@aaronkvanmeerten on which which metric they would do this? "Right now the instances are intended to just shut themselves down (via shell scripts triggered by the sidecar)"

aaronkvanmeerten commented 4 months ago

@aaronkvanmeerten on which which metric they would do this?

They do this in response to a signal from the server. The server selects the least loaded instance, based on either stress level or usage, depending on the type of instance. Right now, stress level is supported for most components except Jibri, which uses a binary on or off state