Canary service never gets any traffic

13567436138 commented 2 years ago

Describe the bug

A clear and concise description of what the bug is. Please provide the Canary definition and Flagger logs. Canary service never gets any traffic

To Reproduce

I have the config:

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: bookinfo
spec:
  revertOnDeletion: true
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: reviews-v1
  service:
    port: 9080
    gateways:
    - istio/bookinfo-gateway
    hosts:
    - "*"
  analysis:
    interval: 1m
    threshold: 10
    maxWeight: 50
    stepWeight: 20
    metrics:
      - name: "my metric"
        thresholdRange:
          min: 0
          max: 10
        interval: 1m
        query: |
          100 - sum(
               rate(
                    istio_requests_total{
                      reporter="destination",
                      destination_workload_namespace="{{ namespace }}",
                      destination_workload="{{ target }}",
                      response_code!="404"
                    }[{{ interval }}]
                )
            )
            /
            sum(
                rate(
                    istio_requests_total{
                      reporter="destination",
                      destination_workload_namespace="{{ namespace }}",
                      destination_workload="{{ target }}"
                    }[{{ interval }}]
                )
            ) * 100

apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
  name: bookinfo
spec:
  revertOnDeletion: true
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: reviews-v1
  service:
    port: 9080
    gateways:
    - istio/bookinfo-gateway
    hosts:
    - "*"
  analysis:
    interval: 1m
    threshold: 10
    maxWeight: 50
    stepWeight: 20
    metrics:
      - name: "my metric"
        threshold: 10
        interval: 1m
        query: |
          100 - sum(
               rate(
                    istio_requests_total{
                      reporter="destination",
                      destination_workload_namespace="{{ namespace }}",
                      destination_workload="{{ target }}",
                      response_code!="404"
                    }[{{ interval }}]
                )
            )
            /
            sum(
                rate(
                    istio_requests_total{
                      reporter="destination",
                      destination_workload_namespace="{{ namespace }}",
                      destination_workload="{{ target }}"
                    }[{{ interval }}]
                )
            ) * 100

the describe canary have the event,but the traffic never promote:

Events:
  Type     Reason  Age                From     Message
  ----     ------  ----               ----     -------
  Warning  Synced  24m                flagger  reviews-v1-primary.istio not ready: waiting for rollout to finish: observed deployment generation less than desired generation
  Normal   Synced  23m (x2 over 24m)  flagger  all the metrics providers are available!
  Normal   Synced  23m                flagger  Initialization done! bookinfo.istio

Expected behavior

A clear and concise description of what you expected to happen.

Additional context

Flagger version: 1.19.0
Kubernetes version:
[root@node01 ~]# kubectl version --short Client Version: v1.21.0 Server Version: v1.21.0
Service Mesh provider: istio
[root@node01 ~]# istioctl version client version: 1.12.0-alpha.5 control plane version: 1.12.0-alpha.5 data plane version: 1.12.0-alpha.5 (15 proxies)
Ingress provider:

amrmotan commented 2 years ago

You might have the same issue I am having now. Note that in my case I am using ASM which is based on istio. After some debugging, I found that the "reporter" comes with the value "source" instead of "destination"

@stefanprodan I wonder if there is any way to override the queries of the built-in metrics, that would be nice. As a workaround, I think we can depend on a custom metric with the same builtin query but change the reported values to "source"

amrmotan commented 2 years ago

This issue looks similar as well https://github.com/fluxcd/flagger/issues/1005

fluxcd / flagger