bookingcom / shipper

Kubernetes native multi-cluster canary or blue-green rollouts using Helm
Apache License 2.0
734 stars 39 forks source link

Validate Service's .spec.selector when shifting traffic #257

Open juliogreff opened 4 years ago

juliogreff commented 4 years ago

Shipper now uses Endpoints to check for traffic progression (#238), which is an improvement over just checking that pods got their labels. If pods don't make it into the Endpoints object, shipper will no longer declare the TrafficTarget as Ready, but we're still lousy at relaying to the user what could possibly be wrong if the pods just don't make it.

There are two obvious reasons why that could happen. The first one, where we can't do much about, is that kubernetes is still busy taking the label shifting into account, but things will eventually converge.

The other reason, where things won't eventually converge, is when the app's Service object has a selector that will never match any pods. If that's the case, shipper will just say "0 out of X pods are not ready", which makes no sense (my traffic target should be ready if nothing is not ready!). To prevent that, the traffic controller should validate that the service has a selector that will match pods, and report the appropriate error through a condition in the TrafficTarget, before doing any actual traffic shifting.