Canary Releases

[oskoi]

Is your feature request related to a problem? Please describe. Ability to divide traffic between releases and/or weights

Describe the solution you'd like It would be a nice feature that simplify deployment of backends

[kpacha]

@oskoi can you elaborate a little bit more? because you can already use the SD layer to decide how to distribute the request among your backends.

ex: 90% to backend A and 10% to B with static SD

"host": [
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backenda.domain.tld",
    "http://backendb.domain.tld",
]

in order to avoid changing your config every time you want to alter the load distribution, you can use other SD and hide the logic there

[oskoi]

@kpacha As for me, looks like a hack or a tradeoff :) I don't have precise control over traffic distribution. I can not send 1% of traffic to one backend.

There are three cases:

1. Distribute using weights (backendA weight: 20, backendB weight: 140, backendC weight: 1000). It simplifies adding new backends without editing old ones.

2. Distribute using percentages. It is like previous case but with explicit percentage notation. If I want to deploy a new release, I could reconfigure krakend to send 99% of traffic to the current version and 1% to canary version. It would be nice too if I could duplicate traffic of a certain percentage (ex: duplicate 10% of traffic to backendB).

3. Distribute using a header. Every backend definition have header key and expected value and based on it Gateway can distribute traffic (ex: backendA receive traffic if a request contains a header: A=1, backendB receive traffic if a request contains a header: A=2, backendC receive all other traffic).

[kpacha]

As commented, I think such logic should be managed by the SD. I posted a very simple example just for demonstration, but the clean and proper way to do it is using DNS SRV records.

$ dig _sip._udp.sip.voice.google.com SRV +short
10 1 5060 sip-anycast-1.voice.google.com.
20 1 5060 sip-anycast-2.voice.google.com.

as you can see, every name has a priority and a weight. So far, the priority is ignored but the weight is used to add the required copies of the host (as I did in my static example). So the resolution of _sip._udp.sip.voice.google.com should be ["sip-anycast-1.voice.google.com.", "sip-anycast-2.voice.google.com."]

You can use the weight as an absolute weight or as a percentage, it just depends on the value of the sum of all the weights.

Routing by header is not a recommended strategy but can be implemented using a custom martian modifier or http client plugin

Regarding duplicating the traffic for the canary tests, maybe you could use https://www.krakend.io/docs/backends/shadow-backends/

[oskoi]

Thanks for these tips!

Will the priority be added in the future? I've found merged PR https://github.com/devopsfaith/krakend/pull/309. Thanks!

Maybe it worth to add percentage proportion to duplicate traffic? Or it would be overengineering? Ex: I have a high RPS and don't want to duplicate all traffic to my canary tests.

[kpacha]

@oskoi adding a sampler for the shadow backend sounds great to me. On the other hand, it could make testing the results a little bit harder (since not all the requests served by the actual backends will be tested against the canary). But I think adding a note at the documentation should be enough.

[RussellLuo]

Hi, @kpacha

Per the definition of Canary Release:

When you are happy with the new version, you can start routing a few selected users to it. There are different strategies to choose which users will see the new version: ...

What it means is that the selected users should get the responses from the new version of backends. But by design, the responses from the shadow backends will be ignored. So I have two questions:

1. Can we really use Traffic shadowing for canary releases?

   Regarding duplicating the traffic for the canary tests, maybe you could use https://www.krakend.io/docs/backends/shadow-backends/

2. If we can not, what's the recommended way for canary releases?

   Routing by header is not a recommended strategy but can be implemented using a custom martian modifier or http client plugin

[kpacha]

@RussellLuo , I think there are 2 important parts in that Fowler's quote:

1. knowing when you're happy with the new version
2. adding the new instances to the backend pool

the first one can be done with the 'traffic shadowing'. the second, with the SD.

[RussellLuo]

Thanks for your reply!

I agree with you. By using the SD, we can implement some weight-based strategy as you described above.

Fowler also mentioned some other approachs like choosing users based on their profile (e.g. userId)：

There are different strategies to choose which users will see the new version: a simple strategy is to use a random sample; some companies choose to release the new version to their internal users and employees before releasing to the world; another more sophisticated approach is to choose users based on their profile and other demographics.

Maybe we can implement this approach by writing plugins or middleware? Or any other suggestions?

[jan4984]

Yes I need this too. May krakend add this?

[jan4984]

FYI https://github.com/jan4984/krakend/commit/60d1441f1d7935fe81c5e4775238eac187418919. Then a middleware can be made, returning ErrIgnoreResult for users not go through some backend.

[github-actions[bot]]

Hi, thank you for bringing this issue to our attention.

Many factors influence our product roadmaps and determine the features, fixes, and suggestions we implement. When deciding what to prioritize and work on, we combine your feedback and suggestions with insights from our development team, product analytics, research findings, and more.

This information, combined with our product vision, determines what we implement and its priority order. Unfortunately, we don't foresee this issue progressing any further in the short-medium term, and we are closing it.

While this issue is now closed, we continue monitoring requests for our future roadmap, including this one.

If you have additional information you would like to provide, please share.

---

This is an automated comment. Responding to the bot or mentioning it won't have any effect

[github-actions[bot]]

This issue was marked as resolved a long time ago and now has been automatically locked as there has not been any recent activity after it. You can still open a new issue and reference this link.