chartmuseum / charts

ChartMuseum Project Helm Charts
https://artifacthub.io/packages/search?repo=chartmuseum
54 stars 57 forks source link

ChartMuseum caching is not working with dynamic aws credentials #63

Open raysaik opened 1 year ago

raysaik commented 1 year ago

I am deploying Chart museum using helm charts, and below is my configuration file

spec:
  values:
    env:
      open:
        STORAGE: amazon
        STORAGE_AMAZON_BUCKET: xxxx-helm-charts
        STORAGE_AMAZON_PREFIX: xxxx-charts-s3
        STORAGE_AMAZON_REGION: eu-central-1
        AWS_SHARED_CREDENTIALS_FILE: /aws/credentials
        AWS_REGION: eu-central-1
    extraArgs:
      - --cache-interval=15m
    podAnnotations:
        vault.hashicorp.com/agent-inject: "true"
        vault.hashicorp.com/role: "vault-kubernetes"
        vault.hashicorp.com/agent-configmap: 'xxxx-charts-configmap'
        vault.hashicorp.com/agent-inject-containers: "chartmuseum"
        vault.hashicorp.com/secret-volume-path: "/aws"
    serviceAccount:
      create: false
      name: "default"
      automountServiceAccountToken: true

I am using vault aws dynamic secret engine to fetch credentials for connecting to s3. All is working fine, except i am getting this error (as below) in my chartmuseum container logs. The secret is rotated successfully by dynamic secret engine but somehow the chartmuseum code that is calling s3 as per the cache-interval is still using the old credentials. It resolves if we restart it but we do not want to add this restart.

_{"L":"INFO","T":"2023-04-08T19:35:17.293Z","M":"Rebuilding index for tenant","repo":""}
{"L":"ERROR","T":"2023-04-08T19:35:17.371Z","M":"InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.\n\tstatus code: 403, request id: XXXXXXXXXXXXX, host id: 9+****************************************************************************************=","repo":""}_