Changing PAT in secret does not rollout automatically

blackliner commented 5 months ago

Checks

[X] I've already read https://docs.github.com/en/actions/hosting-your-own-runners/managing-self-hosted-runners-with-actions-runner-controller/troubleshooting-actions-runner-controller-errors and I'm sure my issue is not covered in the troubleshooting guide.
[X] I am using charts that are officially provided

Controller Version

Chart: 0.9.1 Controller image: ghcr.io/actions/gha-runner-scale-set-controller:0.9.1

Deployment Method

Helm

Checks

[X] This isn't a question or user support case (For Q&A and community support, go to Discussions).
[X] I've read the Changelog before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes

To Reproduce

1. Install ARC according to tutorial with an expiring PAT
2. Change PAT in the defined `githubConfigSecret` once it is expired
3. See how Listeners fail due to `bad credentials`

Describe the bug

When changing the secret that is used in the scalesets githubConfigSecret, the in-between configs that got generated by the operator do not get updated, and the listener pod keeps failing.

Describe the expected behavior

When the content of a relevant secret is changed, the operator should also regenerate the in-between configmaps and secrets that contain aformentioned secret, and recycle the relevant (listener) PODs

Additional Context

Controller Logs

unrelated

Runner Pod Logs

github-actions[bot] commented 5 months ago

Hello! Thank you for filing an issue.

The maintainers will triage your issue shortly.

In the meantime, please take a look at the troubleshooting guide for bug reports.

If this is a feature request, please review our contribution guidelines.

nikola-jokic commented 5 months ago

Hey @blackliner

Can you please write down exact steps how you updated the secret? Did you update it using helm, or by changing it directly?

blackliner commented 5 months ago

The secret was created manually, not hardcoded in the helm chart (gitops). Then the secret (the object iteslf) was updated to reflect the new PAT.

We did not change to a new secret, the existing one was updated.

nikola-jokic commented 5 months ago

Hey @blackliner,

Changing secret in-place cannot trigger the rollout. We are not watching the secret, and we don't hash its content in order to check for changes. The safest way for you to roll out a new secret is to create a new one, modify the values.yaml file to point to a new secret, update ARC installation and then remove the old secret once it is ready.

nikola-jokic commented 5 months ago

Closing this one, please let me know if you need more information :relaxed:

actions / actions-runner-controller