We can build our own actor provisioning logic based on perf metrics and integrate it with existing autoscaler. Coupling it with a cloud-native kube autoscaler (e.g. cluster-autoscaler for cloud-agnostic or karpenter for AWS) will unlock running fully elastic serverless jobs on the cloud. This will consist of two parts:
Scaling Streaming Actors - depends on streaming metrics reporting task
Scaling On-Demand Actors - depends on on-demand metrics reporting
Umbrella task to keep track of autoscaling mechanism for Volga jobs.
Ray provides a nice built-in autoscaler which automatically provisions nodes based on pending actors' resource requests - https://docs.ray.io/en/latest/cluster/vms/user-guides/configuring-autoscaling.html. It is also integrated with Kubernetes (i.e. ray node == kube pod) - https://docs.ray.io/en/latest/cluster/kubernetes/user-guides/configuring-autoscaling.html.
We can build our own actor provisioning logic based on perf metrics and integrate it with existing autoscaler. Coupling it with a cloud-native kube autoscaler (e.g. cluster-autoscaler for cloud-agnostic or karpenter for AWS) will unlock running fully elastic serverless jobs on the cloud. This will consist of two parts: