[bitnami/harbor] exporter.existingEnvVarsSecret should be core.existingEnvVarsSecret

[jdaln]

Name and Version

bitnami/harbor 21.4.6

What architecture are you using?

amd64

What steps will reproduce the bug?

1. Create the following resource:

   
   apiVersion: v1
   kind: Secret
   metadata:
   name: harbor-secret-vars
   namespace: harbor
   type: Opaque
   stringData:
   CSRF_KEY: "YxBzUfRn+Wa9hTVDJzLYDEH0W9syZuodXCmPwo6POYw="
   HARBOR_ADMIN_PASSWORD: "12345678Test"
   POSTGRESQL_PASSWORD: "secure-postgres-password"
   REGISTRY_CREDENTIAL_USERNAME: "registryuser"
   REGISTRY_CREDENTIAL_PASSWORD: "passwordfortest"

2. Install the chart with the values.yaml below overriding the default values of the upstream values.yaml

### Are you using any custom parameters or values?

```yaml
harbor:

  nameOverride: "harbor"

  externalURL: https://core.harbor.example.com

  logLevel: warn

  exposureType: ingress

  service:
    ports:
      http: 80
      https: 443

  core:
    existingEnvVarsSecret: "harbor-secret-vars"

  # exporter: # will be removed when and if https://github.com/bitnami/charts/pull/27451 is merged
  #   existingEnvVarsSecret: "harbor-secret-vars"

  trivy:
    existingEnvVarsSecret: "harbor-secret-vars"

  jobservice:
    existingEnvVarsSecret: "harbor-secret-vars"

  ingress:
    core:
      ingressClassName: "nginx"
      pathType: Prefix
      hostname: core.harbor.example.com
      annotations:
        cert-manager.io/cluster-issuer: "letsencrypt-prod"
        nginx.ingress.kubernetes.io/proxy-body-size: "0"     
      tls:
        - hosts:
          - core.harbor.example.com
          secretName: core.harbor.example.com

  persistence:
    enabled: true
    resourcePolicy: "keep"
    persistentVolumeClaim:
      registry:
        existingClaim: ""
        storageClass: ""
        subPath: ""
        accessModes:
          - ReadWriteOnce
        size: 20Gi
        annotations: {}
        selector: {}
      jobservice:
        existingClaim: ""
        storageClass: ""
        subPath: ""
        accessModes:
          - ReadWriteOnce
        size: 1Gi
        annotations: {}
        selector: {}
      trivy:
        storageClass: ""
        accessModes:
          - ReadWriteOnce
        size: 5Gi
        annotations: {}
        selector: {}

  postgresql:
    enabled: true
    image:
      registry: docker.io
      repository: bitnami/postgresql
      tag: 13.15.0-debian-12-r2
      digest: ""
    auth:
      enablePostgresUser: true
      postgresPassword: not-secure-database-password
      existingSecret: ""
    architecture: standalone
    primary:
      extendedConfiguration: |
        max_connections = 1024
      initdb:
        scripts:
          initial-registry.sql: |
            CREATE DATABASE registry ENCODING 'UTF8';
            \c registry;
            CREATE TABLE schema_migrations(version bigint not null primary key, dirty boolean not null);
      resourcesPreset: "nano"
      resources: {}

  metrics:
    enabled: true
    path: /metrics
    serviceMonitor:
      enabled: false
      namespace: ""
      interval: ""
      scrapeTimeout: ""
      labels: {}
      selector: {}
      relabelings: []
      metricRelabelings: []
      honorLabels: false
      jobLabel: ""

What is the expected behavior?

The cluster should deploy with the correct passwords and environment variable set. It does but a container goes into crashloopbackoff

What do you see instead?

exporter pods throw the following error and the deployment cannot continue

2024-06-19T16:20:38Z [INFO] [/common/dao/base.go:67]: Registering database: type-PostgreSQL host-harbor-postgresql port-5432 database-registry sslmode-"disable"
[ORM]2024/06/19 16:20:38 register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))
2024-06-19T16:20:38Z [FATAL] [/cmd/exporter/main.go:56]: failed to initialize database: register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))

Additional information

I would do a PR but I noticed that this bug is not so straightforward and would like to let the Bitnami team handle it since I do not have time to set up a testing environment to conduct further testing.

exporter.existingEnvVarsSecret should be core.existingEnvVarsSecret in the templates but the templating logic also needs to be fixed because only fixing exporter.existingEnvVarsSecret will result to

2024-06-19T16:20:38Z [INFO] [/common/dao/base.go:67]: Registering database: type-PostgreSQL host-harbor-postgresql port-5432 database-registry sslmode-"disable"
[ORM]2024/06/19 16:20:38 register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))
2024-06-19T16:20:38Z [FATAL] [/cmd/exporter/main.go:56]: failed to initialize database: register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))

For a start, please refer to the comments on https://github.com/bitnami/charts/pull/27451

[migruiz4]

Hi @jdaln,

After reviewing the harbor-exporter logic, it seems that indeed exists an issue when the value core.existingEnvVarsSecret is provided.

I have submitted this PR https://github.com/bitnami/charts/pull/27851 fixing this issue, which replicates the changes in your PR https://github.com/bitnami/charts/pull/27451.

I see your PR was canceled by a DCO issue, so if you'd like to reopen your contribution but you need help with DCO please let me know.

[jdaln]

Hi @migruiz4 ,

Thank you for checking this issue and fixing it.

[migruiz4]

PR merged @jdaln, please give it a try and let me know if it works for you!

[jdaln]

@migruiz4 I just tested this and I get the error below:

➜  harbor git:(feat/harbor-container-registry) ✗ kubectl logs harbor-exporter-6496d79cd6-bzcd9 -n harbor
harbor-exporter 12:18:08.02 INFO  ==> 
harbor-exporter 12:18:08.11 INFO  ==> Welcome to the Bitnami harbor-exporter container
harbor-exporter 12:18:08.12 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
harbor-exporter 12:18:08.21 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
harbor-exporter 12:18:08.21 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
harbor-exporter 12:18:08.31 INFO  ==> 
harbor-exporter 12:18:08.31 INFO  ==> ** Starting harbor-exporter setup **
harbor-exporter 12:18:08.32 INFO  ==> No custom certificates were installed in the system
harbor-exporter 12:18:08.41 INFO  ==> ** harbor-exporter setup finished! **

harbor-exporter 12:18:09.31 INFO  ==> ** Wait for database connection **
harbor-exporter 12:18:09.33 INFO  ==> ** Starting harbor-exporter **
2024-07-12T12:18:09Z [INFO] [/common/dao/base.go:67]: Registering database: type-PostgreSQL host-harbor-postgresql port-5432 database-registry sslmode-"disable"
[ORM]2024/07/12 12:18:10 register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))
2024-07-12T12:18:10Z [FATAL] [/cmd/exporter/main.go:56]: failed to initialize database: register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))

Similar to what I mentioned here https://github.com/bitnami/charts/pull/27451#issuecomment-2179099458

I also have this:

  harbor git:(feat/harbor-container-registry) ✗ kubectl logs harbor-core-57b5fbd748-hfvcf -n harbor
 12:46:50.23 INFO  ==> 
 12:46:50.24 INFO  ==> Welcome to the Bitnami harbor-core container
 12:46:50.24 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
 12:46:50.25 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
 12:46:50.26 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
 12:46:50.26 INFO  ==> 
 12:46:50.27 INFO  ==> ** Starting harbor-core setup **
harbor-core 12:46:50.31 INFO  ==> Validating Core settings...
harbor-core 12:46:50.41 INFO  ==> No custom certificates were installed in the system
 12:46:50.41 INFO  ==> ** harbor-core setup finished! **

harbor-core 12:46:50.43 INFO  ==> ** Starting harbor-core **
2024/07/12 12:46:50.532 [D]  init global config instance failed. If you do not use this, just ignore it.  open conf/app.conf: no such file or directory
2024-07-12T12:46:50Z [INFO] [/controller/artifact/annotation/parser.go:85]: the annotation parser to parser artifact annotation version v1alpha1 registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cncf.helm.config.v1+json registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.cnab.manifest.v1 registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.oci.image.index.v1+json registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.distribution.manifest.list.v2+json registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.distribution.manifest.v1+prettyjws registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.oci.image.config.v1+json registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.docker.container.image.v1+json registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.goharbor.harbor.sbom.v1 registered
2024-07-12T12:46:50Z [INFO] [/controller/artifact/processor/processor.go:59]: the processor to process media type application/vnd.wasm.config.v1+json registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/native/adapter.go:36]: the factory for adapter docker-registry registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/aliacr/adapter.go:44]: the factory for adapter ali-acr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/awsecr/adapter.go:44]: the factory for adapter aws-ecr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/azurecr/adapter.go:29]: Factory for adapter azure-acr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/dockerhub/adapter.go:40]: Factory for adapter docker-hub registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/dtr/adapter.go:36]: the factory of dtr adapter was registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/githubcr/adapter.go:43]: the factory for adapter github-ghcr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/gitlab/adapter.go:33]: the factory for adapter gitlab registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/googlegcr/adapter.go:37]: the factory for adapter google-gcr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/huawei/huawei_adapter.go:40]: the factory of Huawei adapter was registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/jfrog/adapter.go:42]: the factory of jfrog artifactory adapter was registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/quay/adapter.go:53]: the factory of Quay adapter was registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/tencentcr/adapter.go:55]: the factory for adapter tencent-tcr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/volcenginecr/adapter.go:40]: the factory for adapter volcengine-cr registered
2024-07-12T12:46:50Z [INFO] [/pkg/reg/adapter/harbor/adaper.go:31]: the factory for adapter harbor registered
2024-07-12T12:46:50Z [INFO] [/core/controllers/base.go:159]: Config path: /etc/core/app.conf
2024-07-12T12:46:50Z [INFO] [/core/main.go:145]: initializing cache ...
2024-07-12T12:46:50Z [FATAL] [/core/main.go:156]: failed to initialize cache: cache type  not support

[jdaln]

The second error has disappeared in a a subsequent deployment but the first error is definitely still the blocker.

[migruiz4]

Hi @jdaln,

The error refers to an issue with the password provided in your env vars secret:

2024-07-12T12:18:10Z [FATAL] [/cmd/exporter/main.go:56]: failed to initialize database: register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))

Because you are providing your own env vars secret it is not possible to determine if it is an issue with the chart or with the secrets you provided.

I would recommend using the chart secrets as reference to create yours:

• Core secret env vars: https://github.com/bitnami/charts/blob/main/bitnami/harbor/templates/core/core-secret-envvars.yaml
• Exporter CM env vars: https://github.com/bitnami/charts/blob/main/bitnami/harbor/templates/exporter/exporter-cm-envvars.yaml

As you can see, the exporter settings will be taken from different sources:

• HARBOR_DATABASE_PASSWORD points to POSTGRESQL_PASSWORD in core secret env vars.
• HARBOR_ADMIN_PASSWORD points to HARBOR_ADMIN_PASSWORD in core secret env vars.
• The rest of the settings must be provided in the exporter cm env vars.

NOTE: variables in the core secret env vars must be base64 encoded.

[jdaln]

@migruiz4

Thank you for your time!

Indeed I should add HARBOR_DATABASE_PASSWORD but this alone does solve the problem. I will try to be more explicit on my further testing I did because I have assumed full familiarity with the codebase in my previous comments.

There is the following inconsistency in the doc / scripts. https://github.com/bitnami/charts/blob/c14c3f9b47ad12a2ee5491ab66b3de26b27a40c1/bitnami/harbor/values.yaml#L1474 If POSTGRESQL_PASSWORD is required, then HARBOR_DATABASE_PASSWORD and postgres-password should also be required.

This, as well as values.yml containing the following under postgres:, for example.

    auth:
      existingSecret: "harbor-secret-vars"

I have to admit that, as a user, I would perfectly be happy if the password would be generated for me so POSTGRESQL_PASSWORD should probably not be required, unless external DB is in use.

And... for the user provided password, unfortunately, even with all the above, setting the Postgres password still does not work.

I get the following error:

$ kubectl logs harbor-exporter-657cb4d677-rdzv4 -n harbor
harbor-exporter 16:20:08.02 INFO  ==> 
harbor-exporter 16:20:08.11 INFO  ==> Welcome to the Bitnami harbor-exporter container
harbor-exporter 16:20:08.20 INFO  ==> Subscribe to project updates by watching https://github.com/bitnami/containers
harbor-exporter 16:20:08.21 INFO  ==> Submit issues and feature requests at https://github.com/bitnami/containers/issues
harbor-exporter 16:20:08.21 INFO  ==> Upgrade to Tanzu Application Catalog for production environments to access custom-configured and pre-packaged software components. Gain enhanced features, including Software Bill of Materials (SBOM), CVE scan result reports, and VEX documents. To learn more, visit https://bitnami.com/enterprise
harbor-exporter 16:20:08.22 INFO  ==> 
harbor-exporter 16:20:08.31 INFO  ==> ** Starting harbor-exporter setup **
harbor-exporter 16:20:08.32 INFO  ==> No custom certificates were installed in the system

harbor-exporter 16:20:08.41 INFO  ==> ** harbor-exporter setup finished! **
harbor-exporter 16:20:09.21 INFO  ==> ** Wait for database connection **
harbor-exporter 16:20:09.21 INFO  ==> ** Starting harbor-exporter **
2024-07-17T16:20:09Z [INFO] [/common/dao/base.go:67]: Registering database: type-PostgreSQL host-harbor-postgresql port-5432 database-registry sslmode-"disable"
[ORM]2024/07/17 16:20:09 register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))
2024-07-17T16:20:09Z [FATAL] [/cmd/exporter/main.go:56]: failed to initialize database: register db Ping `default`, failed to connect to `host=harbor-postgresql user=postgres database=registry`: server error (FATAL: password authentication failed for user "postgres" (SQLSTATE 28P01))

And, yes, I did check that the secrets are ok and base64:

$ kubectl -n harbor get secrets -o yaml | grep "password"
    postgres-password: c2VjdXJlLXBvc3RncmVzLXBhc3N3b3Jk

and

$ kubectl -n harbor get secrets -o yaml | grep "PASSWORD"
    REGISTRY_REDIS_PASSWORD: ""
    HARBOR_ADMIN_PASSWORD: MTIzNDU2NzhUZXN0
    HARBOR_DATABASE_PASSWORD: c2VjdXJlLXBvc3RncmVzLXBhc3N3b3Jk
    POSTGRESQL_PASSWORD: c2VjdXJlLXBvc3RncmVzLXBhc3N3b3Jk
    REGISTRY_CREDENTIAL_PASSWORD: cGFzc3dvcmRmb3J0ZXN0

I use the following that I kubectl apply -f (of course that automatically takes care of base64enc nowadays):

apiVersion: v1
kind: Secret
metadata:
  name: harbor-secret-vars
  namespace: harbor
type: Opaque
stringData:
  CSRF_KEY: "Whateverthiscanbe"
  HARBOR_ADMIN_PASSWORD: "12345678Test"
  HARBOR_DATABASE_PASSWORD: "secure-postgres-password"
  POSTGRESQL_PASSWORD: "secure-postgres-password"
  REGISTRY_CREDENTIAL_USERNAME: "registryuser"
  REGISTRY_CREDENTIAL_PASSWORD: "passwordfortest"
  postgres-password: "secure-postgres-password"

and the following values.yaml:

harbor:
  nameOverride: "harbor"
  externalURL: https://harbor.example.com
  logLevel: warn
  exposureType: ingress
  service:
    ports:
      http: 80
      https: 443

  core:
    existingEnvVarsSecret: "harbor-secret-vars"

  trivy:
    existingEnvVarsSecret: "harbor-secret-vars"

  jobservice:
    existingEnvVarsSecret: "harbor-secret-vars"

  ingress:
    core:
      ingressClassName: "nginx"
      pathType: Prefix
      hostname: harbor.example.com
      annotations:
        cert-manager.io/cluster-issuer: "letsencrypt-prod"
        nginx.ingress.kubernetes.io/proxy-body-size: "0"     
      tls:
        - hosts:
          - harbor.example.com
          secretName: harbor.example.local

  persistence:
    enabled: true
    resourcePolicy: "keep"
    persistentVolumeClaim:
      registry:
        existingClaim: ""
        storageClass: ""
        subPath: ""
        accessModes:
          - ReadWriteOnce
        size: 20Gi
        annotations: {}
        selector: {}
      jobservice:
        existingClaim: ""
        storageClass: ""
        subPath: ""
        accessModes:
          - ReadWriteOnce
        size: 1Gi
        annotations: {}
        selector: {}
      trivy:
        storageClass: ""
        accessModes:
          - ReadWriteOnce
        size: 5Gi
        annotations: {}
        selector: {}

  postgresql:
    enabled: true
    auth:
      existingSecret: "harbor-secret-vars"

  metrics:
    enabled: true
    path: /metrics
    serviceMonitor:
      enabled: false
      namespace: ""
      interval: ""
      scrapeTimeout: ""
      labels: {}
      selector: {}
      relabelings: []
      metricRelabelings: []
      honorLabels: false
      jobLabel: ""

The reason why I have not proceeded with any further MR is that this needs a decision from your team on how to proceed with the issue. Do you want to provide user-configurable password or not? If so, how do you want to implement it?

[jdaln]

@migruiz4 I have opened a specific issue regarding the DB password https://github.com/bitnami/charts/issues/28188