Changes the K8s-cronjob to a sidecar

[sistason]

Fixes #204 by migrating the r10k cronjobs to sidecars The sidecar is using crond, while still respecting all values.yaml options from the cronjob

[Xtigyro]

Hey @sistason - thank you very much for the nice contribution!

We agree that this is the way to go though it will be utilized usefully only when the support for multiple Puppet masters running on different K8s nodes lands in the chart.

We're gonna review the PR in the coming days. Thank you once more! It's appreciated!

[Xtigyro]

Hey @sistason - I tested it today but I couldn't make it work. Firstly, there's nothing in the directory where the Puppet Control repo should be synced:

/opt # ls -ltrah /etc/puppetlabs/code/environments/production/
total 8
drwxr-xr-x    3 puppet   ping        4.0K Apr 11 19:08 ..
drwxr-x---    2 puppet   ping        4.0K Apr 11 19:08 .

And it appears not to be able to clone a remote repo:

/home/puppet # /opt/r10k.cronjob.sh
ERROR    -> Unable to determine current branches for Git source 'puppet_repo' (/etc/puppetlabs/code/environments)
Original exception:
Command exited with non-zero exit code:
Command: git clone --mirror git@github.com:Xtigyro/puppet-control-repo.git /etc/puppetlabs/code/r10k_cache/git@github.com-Xtigyro-puppet-control-repo.git
Stderr:
Cloning into bare repository '/etc/puppetlabs/code/r10k_cache/git@github.com-Xtigyro-puppet-control-repo.git'...
Host key verification failed.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
Exit code: 128
1

Secondly, using root as the run-user is an anti-pattern and we try to avoid it. Lastly, we've updated the chart to support Helm v3. Some refactoring is required.

Do you think you can try to work on those points? Or can we work together?

Let me know please - so we organize our efforts efficiently. You can msg me in the Puppet Community Slack workspace.

And thanks once again for the contribution!

[Xtigyro]

@sistason I've started the work here: https://github.com/Xtigyro/puppetserver-helm-chart/tree/v310

[Xtigyro]

@underscorgan @sistason It appears we can achieve running r10k at a certain schedule without being the superuser. It can be done by using supercronic. The one small thing that is missing is to optionally exit when the job that it run failed.

An issue has been filed to the creators of that awesome tool: https://github.com/aptible/supercronic/issues/60

[Xtigyro]

For reference purposes - the PR which adds supercronic to the r10k official Docker image. -- https://github.com/puppetlabs/r10k/pull/1043

[sistason]

I won't be able to put much time in here, also since I have no idea of puppet, just K8s...

• The cronjob-sidecar just uses git/r10k, I cannot help with debugging there, but will ask around
• for helm3/securityContext I'll look at what you changed lately and adapt to that

[Xtigyro]

I won't be able to put much time in here, also since I have no idea of puppet, just K8s...

* The cronjob-sidecar just uses git/r10k, I cannot help with debugging there, but will ask around

* for helm3/securityContext I'll look at what you changed lately and adapt to that

No worries - the issues have been overcome. Thanks for the reply and your time.

CC - @slconley @underscorgan

[sistason]

The issues overcome? via supercronic?

Could you tell me again what I can contribute here, so we're on the same page? :) If there is nothing left, that's also okay ;)

[Xtigyro]

The issues overcome? via supercronic?

Could you tell me again what I can contribute here, so we're on the same page? :) If there is nothing left, that's also okay ;)

Via supercronic and changing the code. You can definitely test it - that would be useful! -- https://github.com/Xtigyro/puppetserver-helm-chart/tree/edge Please for the r10k Docker image use (as the PR in the r10k has not been merged just yet):

r10k:
  image: xtigyro/r10k
  tag: 0.2

[kubicgruenfeld]

Via supercronic and changing the code. You can definitely test it - that would be useful! -- https://github.com/Xtigyro/puppetserver-helm-chart/tree/v310 Please for the r10k Docker image use (as the PR in the r10k has not been merged just yet):

i did test the code in the v310 branch and the r10k image but we have still the same issue. the cronjob tries to attach a pvc which is already bound to the puppetserver.

[Xtigyro]

Via supercronic and changing the code. You can definitely test it - that would be useful! -- https://github.com/Xtigyro/puppetserver-helm-chart/tree/v310 Please for the r10k Docker image use (as the PR in the r10k has not been merged just yet):

i did test the code in the v310 branch and the r10k image but we have still the same issue. the cronjob tries to attach a pvc which is already bound to the puppetserver.

@kubicgruenfeld Are you sure you've tested the code from that branch? There's no cronjob in it.

[kubicgruenfeld]

Maybe I missed something. Will check again.

[kubicgruenfeld]

@Xtigyro i tested again and you were right, i was on the wrong branch.

But there are still some bits not working as expected. I have the configmap r10k-code-config but the mount to /etc/puppetlabs/puppet/r10k.yaml does not seem to be working. I had a look through the code and as far as i can tell, this should work.

And if running ~/.r10k_code_cronjob.out i also get this, which seems strange:

...
Running /docker-entrypoint.d/70-set-dns-alt-names.sh
Running /docker-entrypoint.d/80-ca.sh
Running /docker-entrypoint.d/85-setup-storeconfigs.sh
Running /docker-entrypoint.d/90-consul.sh
puppetserver: 'deploy' is not a puppetserver command. See 'puppetserver --help'.

[Xtigyro]

@kubicgruenfeld Could you please verify whether a Puppet Agent can connect and deploy your Puppet/Hiera code?

There's a section in the README which you can literally copy-paste to test it, too: https://github.com/Xtigyro/puppetserver-helm-chart/tree/v310#testing-the-deployed-chart-resources

[Xtigyro]

@kubicgruenfeld In the meantime, I'll try to fix the non-working /etc/puppetlabs/puppet/r10k.yaml - confirmed.

[Xtigyro]

@kubicgruenfeld In the meantime, I'll try to fix the non-working /etc/puppetlabs/puppet/r10k.yaml - confirmed.

@kubicgruenfeld Both issues should be fixed. Could you please re-test?

[Xtigyro]

Found one other small issue which was related to the previous two ones - fixed.

The chart v3.1 should be very soon good to go live!

CC: @underscorgan @slconley

[Xtigyro]

@underscorgan @slconley @kubicgruenfeld @sistason @baurmatt I think we're good to go here. If nobody has found any new bugs - we should be releasing v3.1.0 when the official r10k Docker image bundled with supercronic is released.

[Xtigyro]

@sistason Could you please review the following PR in your fork: https://github.com/syseleven/pupperware/pull/1/

CC: @underscorgan @slconley

[Xtigyro]

Closing this as the chart has been moved to its own GitHub repo at: https://github.com/puppetlabs/puppetserver-helm-chart