goharbor / harbor-operator

Kubernetes operator for Harbor service components
Apache License 2.0
352 stars 108 forks source link

AWS ALB wrong target group health check for core service #967

Open xoanmi opened 2 years ago

xoanmi commented 2 years ago

Problem

When exposing the harbor cluster with an AWS ALB Ingress controller, the Core's target group is using the default health check (/) making AWS mark it as unhealthy even is working properly:

image

NAME                                                      READY   STATUS    RESTARTS   AGE
harborcluster-harbor-harbor-core-57cb4c5857-cwgln         1/1     Running   0          56m
harborcluster-harbor-harbor-jobservice-58bb7b7655-hmwcz   1/1     Running   0          56m
harborcluster-harbor-harbor-portal-5c4d86d966-fpzwg       1/1     Running   0          56m
harborcluster-harbor-harbor-registry-d89d746cf-r9dml      2/2     Running   0          56m

How to reproduce it

apiVersion: goharbor.io/v1beta1
kind: HarborCluster
metadata:
  name: harborcluster
spec:
  expose:
    # Expose core services
    core:
      ingress:
        host: harbor.example.com
        controller: default
        annotations:
          alb.ingress.kubernetes.io/backend-protocol: HTTPS
          alb.ingress.kubernetes.io/certificate-arn: foo
          alb.ingress.kubernetes.io/group.name: harbor
          alb.ingress.kubernetes.io/ip-address-type: ipv4
          alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}, {"HTTP": 80}]'
          alb.ingress.kubernetes.io/load-balancer-name: harbor
          alb.ingress.kubernetes.io/scheme: internal
          alb.ingress.kubernetes.io/ssl-redirect: "443"
          alb.ingress.kubernetes.io/target-type: ip
        ingressClassName: alb
  externalURL: https://harbor.example.com
  harborAdminPasswordRef: harbor-admin-password
  version: "2.5.4"
  internalTLS:
    enabled: true

Proposed solutions

Option 1

Inject the following annotation on the core's service object to allow the ingress controller to create the right health check on the target group in the AWS ALB:

apiVersion: v1
kind: Service
metadata:
  annotations:
    ...
    alb.ingress.kubernetes.io/healthcheck-path: /api/v2.0/ping
    ...

Option 2

Allows the definitions of arbitrary annotations for each service in the harborcluster crd.

cvegagimenez commented 2 years ago

Same problem here!