helmfile / vals

Helm-like configuration values loader with support for various sources
Apache License 2.0
554 stars 74 forks source link

Not able to fetch secret from aws secret manager (cross-accounts) #132

Open xval2307 opened 1 year ago

xval2307 commented 1 year ago

Hello maintainers,

I try to deploy helm secrets + vals on ArgoCD to fetch secrets from aws secrets manager in another account.

    helm:
      fileParameters:
        - name: data.exemple.Key1
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:/SECRET_NAME#/SECRET_KEY_1
        - name: data.exemple.Key2
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:/SECRET_NAME#/SECRET_KEY_2

The arn of mysecret on AWS Console is:

arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME

I haven’t “/” in mysecret name.

And if I remove the “/” in the secret path in helm parameter:

    helm:
      fileParameters:
        - name: data.exemple.Key1
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME#/SECRET_KEY_1
        - name: data.exemple.Key2
          path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME#/SECRET_KEY_2

I have the following error message in ArgoCD:

[helm-secrets] vals error: expand awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME?region=eu-west-3: parse "awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME?region=eu-west-3": invalid port ":SECRET_NAME" after host Error: failed parsing --set-file data: plugin "scripts/run.sh downloader" exited with error

Could you please confirm if this is really the case or it's a misconfiguration of my side?

mumoshu commented 1 year ago

@xval2307 Thanks for reporting! Ahh interesting... so our URL parser is messed up with the last part of your ARN.

Would you try adding a redundant and unused port number after your ARN? That would make the URL parser green and it vals would work as expected as it won't use the information provided via the port part for the AWS SecretsManager mode...

In other words, try changing this:

path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME#/SECRET_KEY_1

to

path: secrets+literal://ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:SECRET_NAME:12345#/SECRET_KEY_1

Notice that I've added :12345 after the ARN part to make the URL parser happy.

fbordallo commented 1 year ago

I'm seeing this problem too. Also, secret names can have forward slashes which could invalidate the workaround.

ref+awssecrets://arn:aws:secretsmanager:eu-west-3:ACCOUNT_ID:secret:secret/name/with/slashes#/SECRET_KEY_1

Thanks for you work!

BabisK commented 1 year ago

I'm facing this issue too, If I add the :12345 after the ARN I get another error when the variable is used:

in ./helmfile.yaml: error during helmfile.yaml.part.1 parsing: template: stringTemplate:22:42: executing "stringTemplate" at <fetchSecretValue>: error calling fetchSecretValue: expand awssecrets://arn:aws:secretsmanager:eu-central-1:xxxxxxx:secret:rds!vvvvvvvvvvvvvvvvvvv:12345: get parameter: ValidationException: Invalid name. Must be a valid name containing alphanumeric characters, or any of the following: -/_+=.@!
        status code: 400, request id: b4f1877d-1113-40f4-97c2-ad12e8e52514
JBOClara commented 10 months ago

Hi @mumoshu ,

We hit this bug and workaround isn't working.

Is there anything I can do to help fixing ?

rikameajay1 commented 6 months ago

Worked for me. Used following format ref+awssecrets://arn:aws:secretsmanager:AWS_REGION:ACCOUNT_ID:secret:/secret/name/with_slashes?region=AWS_REGION Also created secret in AWS Secrets manager as specified in helmfile /secret/name/with_slashes Create secret with "/" at the beginning

anton-prymak commented 5 months ago

It's the same problem for me. Unfortunately, I cannot use"/" at the beginning because of my project limitations.

itssimon commented 1 month ago

I'm also hitting this problem and workarounds aren't working

okouyad commented 1 month ago

Same issue. Given how critical it is, is there some plans to takle this anytime soon ? Else a recommended work around ? This is particularily problematic in situation where there is a cross account access and therefore we need the full arn