hyperledger-bevel / bevel-operator-fabric

Hyperledger Fabric Kubernetes operator - Hyperledger Fabric operator for Kubernetes (v2.3, v2.4 and v2.5, soon 3.0)
https://hyperledger-bevel.github.io/bevel-operator-fabric/
Apache License 2.0
278 stars 93 forks source link

[Gateway API] TLSRoute not created #236

Open Horkyze opened 2 months ago

Horkyze commented 2 months ago

What happened?

I ran

kubectl hlf ca create \
  --storage-class=$STORAGE_CLASS \
  --capacity=1Gi \
  --name=org1-ca \
  --enroll-id=enroll \
  --enroll-pw=enrollpw \
  --gateway-api-hosts $ORG1_CA_HOST \
  --gateway-api-name hlf-gateway \
  --gateway-api-namespace $NAMESPACE \
  --gateway-api-port 443 \
  --namespace=$NAMESPACE

The FabricCA CRD resource was created, but it's missing the Gateway API values such as:

  gatewayApi:
    gatewayName: hlf-gateway
    gatewayNamespace: default
    hosts:
    - org1-ca-example.com
    port: 443

NOTE: passing --output to the kubectl hlf ca create outputs correct values.

Even when manually modifying the FabricCA CRD resource with the gateway values does not work. Lens shows the CRD as updated, but further inspection show that the gateway values are missing.

What did you expect to happen?

I expect the kubectl hlf ca create command to create the TSLRoute CRD.

How can we reproduce it (as minimally and precisely as possible)?

Install traefik, with Gateway API enabled:

ports:
  traefik:
    port: 9000
    expose:
      default: false
    exposedPort: 9000
    protocol: TCP
  web:
    port: 80
    exposedPort: 80
    expose:
      default: true
    protocol: TCP
    redirectTo:
      port: websecure
  websecure:
    port: 443
    exposedPort: 443
    expose:
      default: true
    protocol: TCP
  tlspassthrough:
    port: 8443
    exposedPort: 8443
    expose:
      default: true
    protocol: TCP

ingressRoute:
  dashboard:
    enabled: true

deployment:
  initContainers:
    - name: volume-permissions
      image: busybox:latest
      command: ["sh", "-c", "touch /data/acme.json ; chown 65532:65532 /data/acme.json ; chmod -Rv 600 /data/*"]
      securityContext:
        runAsNonRoot: false
        runAsGroup: 0
        runAsUser: 0
      volumeMounts:
        - name: data
          mountPath: /data

providers:
  # Enable the GatewayAPI provider
  kubernetesGateway:
    enabled: true
    experimentalChannel: true
  kubernetesIngress:
    publishedService:
      enabled: true

gateway:
  enabled: true
  namespacePolicy: All
  listeners:
    tlspassthrough:
      port: 8443
      protocol: TLS
      mode: Passthrough
    web:
      port: 80
      hostname:
      protocol: HTTP

persistence:
  enabled: true
  storageClass: cinder-ssd

certResolvers:
  letsencrypt:
    email: acme@example.com
    tlsChallenge: true
    httpChallenge:
      entryPoint: "web"
    storage: /data/acme.json

logs:
  general:
    format:
    level: INFO
    # noColor: true
  access:
    enabled: false

Anything else we need to know?

contoller logs

1.7259154457291129e+09  INFO    controllers.FabricCA    purge requested for org1-ca
1.7259154458758335e+09  INFO    controllers.FabricCA    CA resource not found. Ignoring since object must be deleted.   {"hlf": "wekeo-eo-dev/org1-ca"}
1.7259155556021981e+09  INFO    controllers.FabricCA    Adding Finalizer for the CA {"hlf": "XXXXXX/org1-ca"}
1.7259155566638715e+09  INFO    controllers.FabricCA    creating 10 resource(s)
1.7259155570278666e+09  INFO    controllers.FabricCA    preparing upgrade for org1-ca
1.7259155579421625e+09  INFO    controllers.FabricCA    performing update for org1-ca
1.7259155582418983e+09  INFO    controllers.FabricCA    creating upgraded release for org1-ca
1.7259155582697067e+09  INFO    controllers.FabricCA    checking 10 resources for changes
1.7259155582748258e+09  INFO    controllers.FabricCA    Patch Secret "org1-ca--ca" in namespace default
1.725915558285192e+09   INFO    controllers.FabricCA    Looks like there are no changes for Secret "org1-ca--msp-cryptomaterial"
1.7259155582936523e+09  INFO    controllers.FabricCA    Looks like there are no changes for Secret "org1-ca--msp-tls-cryptomaterial"
1.7259155583350508e+09  INFO    controllers.FabricCA    Looks like there are no changes for Secret "org1-ca--tls-cryptomaterial"
1.7259155583456178e+09  INFO    controllers.FabricCA    Looks like there are no changes for ConfigMap "org1-ca--ca"
1.7259155583572245e+09  INFO    controllers.FabricCA    Looks like there are no changes for ConfigMap "org1-ca--config-tls"
1.7259155583692648e+09  INFO    controllers.FabricCA    Looks like there are no changes for ConfigMap "org1-ca--config"
1.725915558376979e+09   INFO    controllers.FabricCA    Looks like there are no changes for PersistentVolumeClaim "org1-ca"
1.725915558384629e+09   INFO    controllers.FabricCA    Looks like there are no changes for Service "org1-ca"
1.725915558394765e+09   INFO    controllers.FabricCA    Patch Deployment "org1-ca" in namespace default
1.7259155584249196e+09  INFO    controllers.FabricCA    updating status for upgraded release for org1-ca
time="2024-09-09T20:59:18Z" level=info msg="CA org1-ca in pending status, refreshing state in 10 seconds"
1.725915558509569e+09   INFO    controllers.FabricCA    preparing upgrade for org1-ca
1.725915559445631e+09   INFO    controllers.FabricCA    performing update for org1-ca

Kubernetes version

```console NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME node-stf4a2p2yxax-master-0 Ready master 4d8h v1.23.5 10.0.0.14 xxx Fedora CoreOS 35.20220424.3.0 5.17.4-200.fc35.x86_64 docker://20.10.12 node-stf4a2p2yxax-node-0 Ready 4d8h v1.23.5 10.0.0.234 xxx Fedora CoreOS 35.20220424.3.0 5.17.4-200.fc35.x86_64 docker://20.10.12 node-stf4a2p2yxax-node-1 Ready 4d8h v1.23.5 10.0.0.119 xxx Fedora CoreOS 35.20220424.3.0 5.17.4-200.fc35.x86_64 docker://20.10.12 ```