Provide support for setCanaryScale feature in Simple Canary Deployments. Currently this is only available in Canary w/ Traffic Routing. This is required for JVM based services that need an initial cache creation step to be performed by a single pod in the replica set.
Use Cases
Currently Blue Green deployment stragey allow us to create a single pod in the previewReplicaSet using the parameter previewReplicaCount. This enables the new Revision on previewService to scale the spec.Replicas without receiving any traffic. Our JVM based services need this feature to bring one instance of the new revision that can perform tasks like initial cache creation. The remaining pods in the same replicaSet utilize this cache which helps with a quick startup and lowers the load on database.
When moving form Blue Green to Basic Canary we tried to replicate this feature by using the setWeight feature and adding the initial weight
to 1% ex.
This works really well when you have <=100 pods which is not the case for Production workloads.
Thus we end up having >1 pod trying to perform the expensive cache creation activity thus overloading database.
Questions.
Why not use Canary w/ Traffic routing that has setCanaryScale which lets you scale canary replicas.?
We observed some availability issues with it (#2864 ) and might consider to move to Ping Pong in the long term but as of now
Basic/Simple canary is the best solution as it needs minimal changes within our codebase.
Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.
Summary
Provide support for
setCanaryScale
feature in Simple Canary Deployments. Currently this is only available in Canary w/ Traffic Routing. This is required for JVM based services that need an initial cache creation step to be performed by a single pod in the replica set.Use Cases
Currently Blue Green deployment stragey allow us to create a single pod in the previewReplicaSet using the parameter
previewReplicaCount
. This enables the new Revision onpreviewService
to scale thespec.Replicas
without receiving any traffic. Our JVM based services need this feature to bring one instance of the new revision that can perform tasks like initial cache creation. The remaining pods in the same replicaSet utilize this cache which helps with a quick startup and lowers the load on database.When moving form Blue Green to Basic Canary we tried to replicate this feature by using the
setWeight
feature and adding the initial weight to 1% ex.This works really well when you have <=100 pods which is not the case for Production workloads. Thus we end up having >1 pod trying to perform the expensive cache creation activity thus overloading database.
Questions.
setCanaryScale
which lets you scale canary replicas.? We observed some availability issues with it (#2864 ) and might consider to move to Ping Pong in the long term but as of now Basic/Simple canary is the best solution as it needs minimal changes within our codebase.Message from the maintainers:
Impacted by this bug? Give it a 👍. We prioritize the issues with the most 👍.