Document solution to problem of random passwords change on upgrade (from helm/charts)

scottrigby commented 6 years ago

I was replying in helm/charts#3591 and realized this is a much bigger problem that deserves a charts issue to decide whether we should agree on and document a policy for this across charts.

Can we address this at one of the next charts meetings?

astorath commented 5 years ago

Hi! I'm using the following solution:

Create separate autogen secret resource:

kind: Secret
apiVersion: v1
type: Opaque
metadata: 
name: "consul-autogen"
annotations: 
"helm.sh/hook": "pre-install"
"helm.sh/hook-delete-policy": "before-hook-creation"
data: 
"admin-token": {{ randAlphaNum 24 | quote }}

Create normal secret resource:

kind: Secret
apiVersion: v1
type: Opaque
metadata: 
name: "consul"
data: 
"admin-token": {{ .Values.adminToken | quote }}

Then reference generated secret value or normal secret value depending on settings.

Comments:

this solves any problem with helm delete --purge and helm install (because autogen secret gets replaced in the case).
this is not clean, as after helm delete --purge autogen secret stays. But helm authors don't want to alter hooks behaviour: https://github.com/helm/helm/issues/4434

zhming0 commented 5 years ago

This issue happens on helm 3.0.0-beta.3 too. Might be worthwhile to show this caveat somewhere in the doc?

agronholm commented 5 years ago

This issue happens on helm 3.0.0-beta.3 too

Did you expect this to be fixed? It's not a simple bug, it's a design problem.

vvanouytsel commented 4 years ago

Could a workaround be to use the 'pre-install' hook to create a Secret with a generated password using the 'randAlphaNum' function and a 'pre-delete' hook to make sure the Secret gets deleted if the Release is deleted?

Edit: Seems we can only specify manifest files in hooks, so it seems there might not really be a workaround as long as the hook functionality is not altered.

mattdornfeld commented 4 years ago

I hate to criticize, but this really needs to be solved. This thread is almost two years old. Couldn't this be solved by adding in annotation that will leave a resource unchanged in the event of an upgrade?

PhoneixS commented 4 years ago

I hate to criticize, but this really needs to be solved. This thread is almost two years old. Couldn't this be solved by adding in annotation that will leave a resource unchanged in the event of an upgrade?

There was an attempt but failed in the implementation, you can read more about it in helm/helm#5290 and why it wasn't merged. I think that @cgetzen was thinking on taking it but I'm not sure.

squillace commented 4 years ago

From my perspective, the chart chose to generate the password; but we'd like Helm to do something about that? Shouldn't the chart be written differently? I get wanting helm to be something like a password manager, but doesn't that seem.... wrong?

cgetzen commented 4 years ago

@PhoneixS IMO github.com/helm/helm/pull/5290 doesn't fill most requirements of using helm with random values. If anyone is jumping at the bit, I'd love to see additions to Masterminds/sprig that allow you to set a seed for the random values, which could then be fed in from a chart if we exposed either the InstallDate, or a random value that get's attached to the release on install.

desaintmartin commented 4 years ago

From my perspective, the chart chose to generate the password; but we'd like Helm to do something about that? Shouldn't the chart be written differently? I get wanting helm to be something like a password manager, but doesn't that seem.... wrong?

Once again, do we want Helm to act as a vault? Because that it basically what is needed to solve this issue (even if it already stores the manifests... but that's not a vault). If not, we should not try to handle this case in the first place and recommend that the charts do not generate password for the user (and do not hack with lot'o'hooks to store/get the passwords, complexifying chart maintenance).

squillace commented 4 years ago

Yeah, no, we do not -- at all. Charts should either hard-code dumb pwds so they never change if you're doing dev, or they shouldn't specify anything at all and the status should spit out what to change in order to run properly. I get that charts didn't do the right thing here, that the intent was good. But fixing Helm and not the chart is not the right direction to move, imho. I cannot see any persuasive argument for pressing on this in helm rather than on charts.

nuwang commented 4 years ago

@squillace

I get wanting helm to be something like a password manager, but doesn't that seem.... wrong?

The problem I think is that this issue violates the design intent of Kubernetes secrets. Secrets should store things like database passwords, and be able to store them stably for the desired duration, unless the secret is explicitly overwritten. Otherwise, losing access to the secret means possibly irretrievable data loss. What's being asked for is that helm provide a mechanism to say: "do not overwrite this secret if it already exists". That it can't do this I feel is a design flaw in helm, and not a design flaw in charts.

nuwang commented 4 years ago

Also, the current workaround we are using for this, as others have mentioned before, is to add the following annotations to the Secret, with the attendant caveats:

  annotations:
    # xref: https://github.com/helm/helm/issues/3053
    # xref: https://github.com/helm/helm/pull/5290
    "helm.sh/hook": "pre-install"
    "helm.sh/hook-delete-policy": "before-hook-creation"

But unless 3rd party charts uniformly do this too, password handling is still iffy.

nuwang commented 4 years ago

The other issue is that this comes back to the issue of encapsulation. The chart author is in a better position to determine what values should be secure and can be autogenerated. For example, consider the rabbitMqErlangCookie value in the rabbitmq chart. It is currently randomly generated if not specified. The user of the chart need not ever know what it is as long as the value is stable for the duration of the installation. Forcing the user to deal with this makes something that could have been simple and secure out of the box, now complex and possibly insecure (e.g. poor choice of password).

squillace commented 4 years ago

re:

That it can't do this I feel is a design flaw in helm, and not a design flaw in charts.

Understood. I cannot but disagree. From my POV, kubernetes itself doesn't say or do anything about this issue at all, and neither should Helm. Every time I think about this situation, I think: damn, we wrote charts to do somethign that by design causes problems because it breaks during upgrade. We'd best fix the charts!

If I were writing a chart from scratch, I'd require a pwd, full stop. If I wanted a new feature to do this, I'd write a helm plugin that does https://github.com/helm/charts/issues/5167#issuecomment-583064019 and help everyone use it.

I do not want Helm changing chart behavior en masse because charts are doing something they were designed for for developer usage. It's totally and entirely an anti-pattern for production usage, and to modify the default behavior would potentially open the reverse problem:

if you are running in production with an auto-generated pwd, you need to fish that out and replace it explicitly with something random and trusted.
you suddenly now get generated pwds and you are now encouraged to think that this is just peach although no one has vetted this for production-grade security in the first place, least of all you and your organization. You're just happy no one bothers you.

I cannot support from my own perspective any change that fixes the tool because the charts were created a long time ago when we knew less about usage than we do now.

FWIW, I think your example of the randomly generated value for the rabbitmq cookie is a good example, so long as the system doesn't care if it changes. If it changes, then having randomly generated in the chart is a bad practice, and the chart should be changed.

So, I'm on the opposite side of your argument:

The user of the chart need not ever know what it is as long as the value is stable for the duration of the installation. Forcing the user to deal with this makes something that could have been simple and secure out of the box, now complex and possibly insecure (e.g. poor choice of password).

and believe strongly that that chart is broken for ITS users, and not helm at all. I hasten to add that I definitely appreciate the problem, have experienced it myself, and empathize with the difficulty of modifying the original charts that made this mistake. I just can't agree with fixing "some" charts by modifying the tool that makes use of them. But hey, that's just me.

nuwang commented 4 years ago

@squillace What I gathered from your message is that some people are asking that helm be changed so that a selective group of charts can continue to work as is? In that case, I wholeheartedly agree with what you're saying, it doesn't seem right to change helm and break existing behaviour. I'm not arguing for that at all.

What I'm arguing for is that helm should add features to support this functionality, for some of the afore-mentioned reasons. The key ones are:

So as to provide more secure defaults out of the box
Encapsulate details that are really not relevant to chart users and thus, can simplify the external interfaces of the charts.

If Helm can provide a good random number/password generation routine coupled with the ability to selectively preserve or overwrite resources, such as by adding a new annotation as mentioned here (https://github.com/helm/charts/issues/5167#issuecomment-385633991), that would be a fairly simple path forward?

That way, users need not ever set passwords unless they really want to. They can always go into the k8s secret and look it up, making a simple helm install stable/postgresql secure by default.

llech commented 4 years ago

I've learned the hard way that random generated passwords in helm chart are something to be avoided.

However, the problem is, that they are in many tutorials / examples. Probably because people couldn't see any other point of functions like randAlphaNum then generating random passwords.

What is the purpose of providing the functions that have no reasonable use? You don't need random-generated passwords for development, admin/nimda do work well enough.

agronholm commented 4 years ago

I've learned the hard way that random generated passwords in helm chart are something to be avoided.

Would you like to elaborate? If there's a good reason to never do this, it might help settle this discussion.

danielezer commented 4 years ago

I want to add that the reality is that a lot of charts out there actually use this option and thus, a lot of users encounter it. Off the top of my head, I can say that the postgresql and mongodb charts use this to generate a random password if none was provided. I'm sure that there are many other charts that use the randAlphaNum function to generate random strings, so, instead of trying to educate everyone to stop using it (which is, IMHO, not really feasible), isn't it easier to introduce an opt-in option to keep the value static?

squillace commented 4 years ago

I totally agree that someone might want this ability, and so creating a helm plugin per https://github.com/helm/charts/issues/5167#issuecomment-583064019 to enable it is the way to go, IMHO. There is nothing stopping anyone from doing this right now. And it'll likely be very popular!

I agree with suggestions that this would be useful. There is nothing stopping anyone from creating this functionality right now!

squillace commented 4 years ago

Even better: have it examine charts for random generation, and prompt to keep, replace, and/or generate one and inject it so that generation occurs outside of the release process.....

arm4b commented 4 years ago

Hey, it's not only about randAlphaNum, there is also genPrivateKey which can generate private SSH keys for Helm charts. Is it improper use as well? Also, if secrets generation are not favored in Helm, the situation is the same for any autogenerated values and their state.

I don't think anyone is asking about using Helm as a password manager. It's about handling state for the value.

Extracting some of the thoughts from one of the Helm charts that tried to use autogenerated Helm values: https://github.com/StackStorm/stackstorm-ha/pull/62. Here are discovered problems to be clear:

Helm doesn't provide enough instrumentation to work with autogenerated values and their state. It's not just passwords.

1) It's possible to autogenerate a value, but it will be re-generated every time on Helm upgrade which is undesired 2) It's possible to workaround value generation only once on install and mark it as unmanaged by Helm (so it's not deleted on upgrade), but then there is no way to transition to user-provided data in helm values after that

I've seen other official Helm chart repositories and what they do: allow or autogenerated or predefined secrets without proper transition, meaning if user had autogenerated secret on first install (default), they can't change it to custom with no chance to rotate the secrets afterwards.

The solution for now: a) Don't use autogenerated values at all; b) Hardcode "example/dummy" secrets in Helm values which is more insecure; c) Leave the secrets empty forcing user to fill it, which is a poor user experience. Both Helm core maintainers and Charts maintainers would loose from this situation. We all want better user experience and adoption.

With that missing functionality with autogenerated < > predefined value transition a better solution introduced in Helm itself is something that people are asking.

squillace commented 4 years ago

ANY function can have this behavior, though you're right that most involve secrets. Anything generated by what is essentially an out of band function will create this situation. I remain uninterested in solving this myself and remain of the position that those who want to address this situation have the power to do so already with either a complete implementation, or a plugin. In fact, creating this behavior in a plugin is a great way to vet a solution here generally if it is to be adopted in the future.

llech commented 4 years ago

I've learned the hard way that random generated passwords in helm chart are something to be avoided.

Would you like to elaborate? If there's a good reason to never do this, it might help settle this discussion.

Well, the problem is, that random generated values that they were working on their own.

The secret was always overwritten, even if nothing has changed in the template or values, because helm is unable to detect, that the only 'difference' is the random generated value.

However, the pods using the secret were not started new because helm is unable to detect the fact that the pod is using the secret that has changed.

It created often a situation, when 2 pods, that were using the same secret, were using different versions of the secret, with different passwords etc.

But on the 2nd thought, I'd like to revert what I've stated. The situation above is an example, that it's a bad idea to use helm to generate secrets in the same chart as the deployments using them, because it doesn't work reliable. However, both secrets and random values work fine in install-only scenario, so the solution that might work well is splitting the helm chart to 2 charts.

The first chart contains only secret, and might utilize random functions, to prevent having passwords in the shell history or in the value files. That chart will be normally installed only once, during namespace setup.

The other will use that secrets and will be upgraded in the regular maintanance cycle.

It's not the problem with that functions, but that the documentation is missing to point out the pitfalls of using them.

agronholm commented 4 years ago

The secret was always overwritten, even if nothing has changed in the template or values, because helm is unable to detect, that the only 'difference' is the random generated value.

I would like to point out that this is exactly the problem that this proposal aims to solve: that generated secrets would not be regenerated on subsequent release updates, but unlike pre-install hooks, they would be removed if the release is deleted.

tahmmee commented 4 years ago

This might be a useful workaround. With helm 3.1 there is a lookup function, which means it's possible to skip upgrading of a secret/configmap if it already exists

{{- if not (lookup "v1" "Secret" .Release.Namespace "my-secret") }}
---
apiVersion: v1
kind: Secret

edit: This appears to be working, except I had to reuse a specific attribute of the secret rather than entire resource because helm will delete it if the template is empty...

{{- define "cluster.password" -}}

{{- $secret := (lookup "v1" "Secret" .Release.Namespace "my-secret") -}}
{{- if $secret -}}
{{/* 
   Reusing current password since secret exists
*/}}
{{-  $secret.data.password -}}
{{- else -}}
{{/* 
    Generate new password
*/}}
{{- (randAlpha 6) | b64enc | quote -}}
{{- end -}}

Then elsewhere in secret

---
apiVersion: v1
kind: Secret
data:
    password: {{ template "cluster.password"  .  }}

herrberk commented 4 years ago

Here is an alternative (requires Helm v3) to tahmmee's solution, in which the generated values are stored in template variables and if the resource already exists, the existing values are used:

{{- $mongoRoot := (randAlpha 16) | b64enc | quote }}
{{- $mongoReplicaKey := (randAlpha 64) | b64enc | quote }}

{{- $secret := (lookup "v1" "Secret" .Release.Namespace "mongo-keys") }}
{{- if $secret }}
{{- $mongoRoot = index $secret.data "mongodb-root-password" }}
{{- $mongoReplicaKey = index $secret.data "mongodb-replica-set-key" }}
{{- end -}}

apiVersion: v1
kind: Secret
metadata:
  name: mongo-keys
  namespace: {{ .Release.Namespace }}
type: Opaque
data:
  mongodb-root-password: {{ $mongoRoot}}
  mongodb-replica-set-key: {{ $mongoReplicaKey }}

movergan commented 3 years ago

Please let's have it fixed. It's ok to use workaround for in-house charts, not for 3rd party dependency though.

mjgallag commented 3 years ago

@herrberk Any ideas for rendering $mongoRoot in NOTES.txt on install? Thanks.

aure-olli commented 3 years ago

Hi

I've rewritten kubernetes replicator and added some annotations to deal with this kind of problems: https://github.com/olli-ai/k8s-replicator#use-random-password-generated-by-an-helm-chart

Now can generate a random password with helm, and replicate it only once to another secret thus it won't be change by helm in the future.

apiVersion: v1
kind: Secret
type: Opaque
metadata:
  name: admin-password-source
  annotations:
    k8s-replicator/replicate-to: "admin-password"
    k8s-replicator/replicate-once: "true"
stringData:
  password: {{ randAlphaNum 64 | quote }}

Hope it will help people.

monotek commented 3 years ago

From helm 3.1.0 you should be able to use the lookup function to read a previously created random.password from kubernetes: https://helm.sh/docs/chart_template_guide/functions_and_pipelines/

llech commented 3 years ago

From helm 3.1.0 you should be able to use the lookup function to read a previously created random.password from kubernetes: https://helm.sh/docs/chart_template_guide/functions_and_pipelines/

How can it work, if the docs state, that:

Keep in mind that Helm is not supposed to contact the Kubernetes API Server during a helm template or a helm install|update|delete|rollback --dry-run, so the lookup function will return an empty list (i.e. dict) in such a case.

so exaclty in the cases where that values would be used?

jglick commented 3 years ago

Helm is not supposed to contact the Kubernetes API Server during a helm template or a helm install|update|delete|rollback --dry-run

helm install --dry-run does contact the API server.

If you use this trick from helm template, the chart will not see an existing secret, so it will simply generate a new one each time.

dudicoco commented 3 years ago

@jglick this would still show a diff when using the helm-diff plugin, so still problematic :(

rgembalik commented 3 years ago

Wouldn't a new value for helm.sh/resource-policy be a decent solution? E.g. update to the annotation functionality so that:

helm.sh/resource-policy can accept multiple values (comma-separated, trimmed by helm upon reading so it can be a nice list)
helm.sh/resource-policy can accept no-update value which will stop updates to once created resource.

This will allow you to specify the following helm.sh/resource-policy values:

empty - no special action is taken against the resource, it can be modified and changed freely
keep - resource is not removed upon uninstall
no-update - resource will not be updated upon changes (unless the change explicitly removes this resource policy, this is for nice to have)
keep, no-update, no-update, keep - do not modify or remove the resource once it's created (again, unless update changes this annotation if we feel fancy)

iTaybb commented 3 years ago

As a solution, I've created the following template:

{{/*
Returns a secret if it already in Kubernetes, otherwise it creates
it randomly.
*/}}
{{- define "getOrGeneratePass" }}
{{- $len := (default 16 .Length) | int -}}
{{- $obj := (lookup "v1" .Kind .Namespace .Name).data -}}
{{- if $obj }}
{{- index $obj .Key -}}
{{- else if (eq (lower .Kind) "secret") -}}
{{- randAlphaNum $len | b64enc -}}
{{- else -}}
{{- randAlphaNum $len -}}
{{- end -}}
{{- end }}

Then you can simply configure secrets like:

---
apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
data:
  PASSWORD: "{{ include "getOrGeneratePass" (dict "Namespace" .Release.Namespace "Kind" "Secret" "Name" "my-secret" "Key" "PASSWORD") }}"

mrtndwrd commented 3 years ago

That works great @iTaybb ! I have a suggestion to improve it a bit so you can still change the contents of the secret in the future:

{{- define "getOrGeneratePass" }}
{{- $len := (default 16 .Length) | int -}}
{{- $obj := (lookup "v1" .Kind .Namespace .Name).data -}}
{{- $val := (index $obj .Key) -}}
{{- if $val }}
{{- $val -}}
{{- else if (eq (lower .Kind) "secret") -}}
{{- randAlphaNum $len | b64enc -}}
{{- else -}}
{{- randAlphaNum $len -}}
{{- end -}}
{{- end }}

By checking if $obj .Key has a true-ish value, you can add data to your secret and the new data will be generated while keeping the old data in tact.

I know this wouldn't work if you have empty data in your secrets, but in my case that's not a problem.

saharhagbi commented 3 years ago

this was my solution for this problem. first, I override existingSecret parameter in postgres sub-chart and thus, instruct postgres to use my own secret and not the auto-generated subchart secret.

postgresql:
  existingSecret: "some name"

then, I created my own secret as the following:

apiVersion: v1
kind: Secret
metadata:
  name: "some-name"
type: Opaque
data:
{{- if .Release.IsUpgrade }}
  postgresql-password:  {{ index (lookup "v1" "Secret" .Release.Namespace .Values.postgresql.existingSecret).data "postgresql-password" }}
{{ else }} # install operation
  postgresql-password: {{ randAlphaNum 20 | b64enc }} 
{{ end }}

EDIT:

I had a requirement to give the existingSecret parameter a templating name - {{ .Release.Name }} + "-postgresql". So, It was a little-bit difficult becasue values.yaml forbid using templating, but after some playing I managed to do that with the following assign:

postgresql:
  existingSecret: "{{ .Release.Name }} -postgresql"

And it works! It seems to me that the string value -"{{ .Release.Name }} -postgresql" - parsed as if we had used tpl function on it. Can it be ?

sanzenwin commented 3 years ago

@saharhagbi , thanks, it works.

varungupta19 commented 2 years ago

Same issue here. I resolved it by rolling back to previous release using helm and getting the old password out. Then i upgraded my helm kong release again with this password. And this worked. I used kong helm chart from bitnami and here is the solution mentioned in there articles https://docs.bitnami.com/general/how-to/troubleshoot-helm-chart-issues/

needleshaped commented 2 years ago

Since 1.21 Kubernetes provides an option to set individual Secrets and ConfigMaps as immutable: https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable.

Can it be a solution?

llech commented 2 years ago

Since 1.21 Kubernetes provides an option to set individual Secrets and ConfigMaps as immutable: https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable.

Can it be a solution?

What will happen if Helm will try to delete or modify such immutable secret?

If the request will be gently ignored, it would be the solution.

saharhagbi commented 2 years ago

Since 1.21 Kubernetes provides an option to set individual Secrets and ConfigMaps as immutable: https://kubernetes.io/docs/concepts/configuration/secret/#secret-immutable.

Can it be a solution?

Did you check it locally? If it works, PostgreSQL chart needs to be updated first

eoneoff commented 2 years ago

My workaround is to combine lookup and resource-policy/keep

{{- if not (lookup "v1" "Secret" .Release.Namespace "mysecret") }}
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
  annotations:
    "helm.sh/resource-policy": "keep"
type: Opaque
stringData:
  password: {{ randAlphaNum 24 }}
{{- end }}

Now on first deployment the secret is created. On upgrade the secret is not charted and is marked to be deleted, but the resource policy prevents it.

jglick commented 2 years ago

@eoneoff I think the idiom shown in https://github.com/helm/helm-www/issues/1259#issuecomment-641558251 is preferable in general, as it ensures that other changes to the Secret resource defined by the chart (say, non-Helm-related annotations) are honored when upgrading.

antevens commented 1 year ago

How about a pattern similar to this where a password definition in _helpers.tpl can be referenced multiple times but yields the same random password in an upgrade safe way?

`{{/ Get current/future passwords in upgrade safe way /}} {{- define "myChart.randomPassword" -}} {{- $secret := lookup "v1" "Secret" .Release.Namespace "my-secret" }} {{- if $secret -}} {{- get $secret.data "password" | b64dec -}} {{- else }} {{- get ( $randAlphaNum16 := set . "randAlphaNum16" ( default ( randAlphaNum 16 ) ( get . "randAlphaNum16" ))) "randAlphaNum16" -}} {{- end }} {{- end }}

apiVersion: v1 kind: Secret type: Opaque metadata: name: my-secret data: password: {{ include "myChart.randomPassword" . | b64enc | quote }}

apiVersion: v1 kind: ConfigMap data: config: |- serverpassword: {{ include "myChart.randomPassword" . }} `

peterwwillis commented 1 year ago

I have cobbled together a solution from a couple different solutions. My use case:

Generate a random password for a Postgres install at helm install time
Populate a K8s secret with the random password
Reuse the same randomly-generated password in the same Helm run
Reuse the K8s secret at helm upgrade time (bonus points: overwrite the secret value outside of Helm and Helm won't care)

First you create a function to generate static passwords. This has the neat property of creating a different random password per Release.Name, though of course you could get more granular with it. Props to https://stackoverflow.com/a/75723185/3760330 for the random password generator function:

{{- define "generate_static_password" -}}
{{- /* Create "tmp_vars" dict inside ".Release" to store various stuff. */ -}}
{{- if not (index .Release "tmp_vars") -}}
{{-   $_ := set .Release "tmp_vars" dict -}}
{{- end -}}
{{- /* Some random ID of this password, in case there will be other random values alongside this instance. */ -}}
{{- $key := printf "%s_%s" .Release.Name "password" -}}
{{- /* If $key does not yet exist in .Release.tmp_vars, then... */ -}}
{{- if not (index .Release.tmp_vars $key) -}}
{{- /* ... store random password under the $key */ -}}
{{-   $_ := set .Release.tmp_vars $key (randAlphaNum 20) -}}
{{- end -}}
{{- /* Retrieve previously generated value. */ -}}
{{- index .Release.tmp_vars $key -}}
{{- end -}}

Next you create a function that will try to lookup the value in the K8s secret on upgrade. If it's not an upgrade (assume it's install) it requires two values, which are the name of the secret to create, and the key to store the password as in the secret. Note that lookup() returns an empty state on template or dry-run, so this solution creates a default dict to avoid errors.

{{- define "random_pw_reusable" -}}
  {{- if .Release.IsUpgrade -}}
    {{- $data := default dict (lookup "v1" "Secret" .Release.Namespace .Values.random_pw_secret_name).data -}}
    {{- if $data -}}
      {{- index $data .Values.random_pw_secret_key | b64dec -}}
    {{- end -}}
  {{- else -}}
    {{- if and (required "You must pass .Values.random_pw_secret_name (the name of a secret to retrieve password from on upgrade)" .Values.random_pw_secret_name) (required "You must pass .Values.random_pw_secret_key (the name of the key in the secret to retrieve password from on upgrade)" .Values.random_pw_secret_key) -}}
      {{- (include "generate_static_password" .) -}}
    {{- end -}}
  {{- end -}}
{{- end -}}

Finally you create your secret:

---
apiVersion: v1
kind: Secret

metadata:
  name: "{{ .Values.random_pw_secret_name }}"
  namespace: "{{ .Release.Namespace }}"

type: Opaque

{{- if .Values.secretData -}}
data:
  {{- range $k, $v := .Values.secretData }}
    "{{ $k }}": "{{ tpl $v $ | b64enc }}"
  {{- end }}
{{- end -}}

values.yaml for the secret:

random_pw_secret_name: db-credentials
random_pw_secret_key:  DB_PASSWORD
secretData:
  DB_HOST: db
  DB_NAME: mydatabase
  DB_USER: postgres
  DB_PASSWORD: '{{- include "random_pw_reusable" . -}}'
  DB_CONNECTION_STRING: 'postgresql://postgres:{{- include "random_pw_reusable" . -}}@db:5432/mydatabase'

As you can see, a new secret will be created with a set of KEY=VALUE pairs. Both DB_CONNECTION_STRING and DB_PASSWORD call the random_pw_reusable function, and both of them will receive the same randomly-generated string at install time, or the value of the DB_PASSWORD at upgrade time.

In this way you can deploy Postgres at install time with a random password, upgrade it and retain the same password, and rotate the password at any time (outside of Helm) yet still retain that rotated password. Resources are managed as normal so a delete/uninstall will work fine. I suppose the only problem is if the secret gets deleted, this won't create a new random password...

helm / helm-www

Document solution to problem of random passwords change on upgrade (from helm/charts) #1259

Comments: