关于kurise-rollout 使用aliyun-alb 作为ingress controller时支持canary-by-header的灰度能力。

版本: kruise-rollout v0.3.0 Kubernetes 版本：1.25.4 ingress controller: aliyun-alb

rollout manifest：

apiVersion: rollouts.kruise.io/v1alpha1
kind: Rollout
metadata:
  name: rollout-with-traffic
  annotations:
    rollouts.kruise.io/rolling-style: partition
  namespace: foo
spec:
  objectRef:
    workloadRef:
      apiVersion: apps/v1
      kind: Deployment
      name: echoserver
  strategy:
    canary:
      steps:
      - replicas: 1
        weight: 5
        pause: {}
      - replicas: 30%
        pause: {}
        matches:
        - headers:
          - name: user-agent
            type: Exact
            value: pc
      - replicas: 60%
        pause: {}
        matches:
        - headers:
          - name: user-agent
            type: Exact
            value: mobile
      - replicas: 100%
        weight: 100
      trafficRoutings:
      - service: echoserver
        ingress:
          name: echoserver
          classType: aliyun-alb

在rollout 的过程当中接入管理入口流量，并且需求根据httpheader 的方式进行对后端灰度，第二阶段通过灰度 user-agent: pc 流量，第三阶段灰度 user-agent: mobile 流量。再此场景下 aliyun-alb 会产生的alb ingress annotation 如下：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/canary: "true"
    alb.ingress.kubernetes.io/canary-by-header: user-agent
    alb.ingress.kubernetes.io/canary-by-header-value: pc
    alb.ingress.kubernetes.io/order: "1"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{},"name":"echoserver","namespace":"foo"},"spec":{"ingressClassName":"alb","rules":[{"host":"echoserver.caocaokeji.cn","http":{"paths":[{"backend":{"service":{"name":"echoserver","port":{"number":80}}},"path":"/apis/echo","pathType":"Exact"}]}}]}}
  creationTimestamp: "2023-03-14T07:53:54Z"
  generation: 1
  name: echoserver-canary
  namespace: foo
...

此规则在执行至第二阶段时对alb 产生全新的rule 规则，并将自己的order ID 该ALB的最前端，并配置ID 为1。此时灰度流量访问正常，如果同时具备多个需要http header的应用同时进行灰度，则会出现aliyun-alb 无法创建 rule的情况。原因为：alb 的 orderID 在宣告的 ingress 当中必须唯一。

# kube-ali-stable get ingress -n foo
NAME                      CLASS   HOSTS                            ADDRESS                                               PORTS   AGE
echoserver                alb     echoserver.example.com          alb-g2jxqzz0nxknlsapxl.cn-hangzhou.alb.aliyuncs.com   80      25h
echoserver-canary         alb     echoserver.example.com          alb-g2jxqzz0nxknlsapxl.cn-hangzhou.alb.aliyuncs.com   80      87m
echoserver-clone          alb     echoserver.example.com          alb-g2jxqzz0nxknlsapxl.cn-hangzhou.alb.aliyuncs.com   80      102m
echoserver-clone-canary   alb     echoserver.example.com                                                                80      117s

# kube-ali-stable get ingress -n foo -o yaml echoserver-clone-canary
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    alb.ingress.kubernetes.io/canary: "true"
    alb.ingress.kubernetes.io/canary-weight: "5"
    alb.ingress.kubernetes.io/order: "1"
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"networking.k8s.io/v1","kind":"Ingress","metadata":{"annotations":{},"name":"echoserver-clone","namespace":"foo"},"spec":{"ingressClassName":"alb","rules":[{"host":"echoserver-clone.caocaokeji.cn","http":{"paths":[{"backend":{"service":{"name":"echoserver-clone","port":{"number":80}}},"path":"/apis/echo","pathType":"Exact"}]}}]}}
  creationTimestamp: "2023-03-14T09:19:46Z"
  generation: 1
  name: echoserver-clone-canary
  namespace: foo
...

如果将 kruise-rollout-controller-manager 内/lua_configuration/trafficrouting_ingress/aliyun-alb.lua 文件 annotations["alb.ingress.kubernetes.io/order"] = "1" 进行删除，则canary ingress 可以正确创建出，并在aliyun-alb上生成全新ruleID ，但客户端进行 curl --silent -H "user-agent: pc " -H "Host: echoserver.example.com" alb-g2jxqzz0nxknlsapxl.cn-hangzhou.alb.aliyuncs.com/apis/echo 时无法进行灰度请求。原因目前查到，由于该canary 的ingress 在aliyun-alb上的生成的 ruleID 后于现有ingress ruleID ，按照aliyun-alb 匹配从上至下匹配规则，由于无法匹配到该header 请求并进行转发。

需求：希望kruise-rollout能够在使用aliyun-alb作为ingress controller的场景下，rollout 对象开启 trafficRoutings 时能支持canary-by-header模式的灰度功能。

openkruise / rollouts

关于kurise-rollout 使用aliyun-alb 作为ingress controller时支持canary-by-header的灰度能力。 #128