search

jenkinsci / configuration-as-code-plugin

Jenkins Configuration as Code Plugin
https://plugins.jenkins.io/configuration-as-code
MIT License
2.68k stars 717 forks source link

Only execute once for initial setup #825

Open reitzmichnicht opened 5 years ago

reitzmichnicht commented 5 years ago

Your checklist for this issue

🚨 Please review the guidelines for contributing to this repository.

Feature Request

Please allow to execute the provided jenkins.yaml only once for the initial setup. We include this file in our Docker container but want to be able to update some configuration without having to deploy a new docker container. These manual changes are lost with each reboot of jenkins. Would be great to have a configuration option allowing to set if the CASC_JENKINS_CONFIG file should be executed on each reboot or only initially.

jetersen commented 5 years ago

Goes against the idea of configuration as code so I might see it as a moot point.

You're welcome to implement and send a PR our way where we can discuss your implementation and needs in more significant details.

reitzmichnicht commented 5 years ago

No this is not really against the idea. We need casc for the initial setup of jenkins with authorization and all the stuff Afterwards we also use a job that updates the casc settings from a git repo on a nightly basis. These updates are gone if the docker container/jenkins is restarted because then the initial config in the docker image is used.

And it is impractical to start/stop a complete container just to add a user to some role.

reitzmichnicht commented 5 years ago

Another solution would be to allow pulling the casc config over http with authentication support or using it from some git repo.

timja commented 5 years ago

You can already pull it with http(s) if you can either:

  1. add a token to the URL to access it, GitHub supports this
  2. configure an access policy that allows unauthenticated access from your jenkins master, something like s3 buckets can do this I believe
Holmistr commented 5 years ago

Actually, I'm facing the same issue. The way I workarounded it is to update the YAML configuration manually each time I do some configuration change, but this is very tedious. I was thinking about exporting the existing YAML configuration e.g. every night, but from what I saw, the "Export configuration" functionality in Configuration as Code section doesn't output really all the configuration. So parts, e.g. for plugins, are missing.

Any suggested way how to do it?

jetersen commented 5 years ago

as @timja said you can already pull config over http with basic auth. Feel free to pick up #665

timja commented 5 years ago

The normal work flow would be not to make manual changes to the master, and apply the change via config as code, that way they aren't out of sync

jetersen commented 5 years ago

@Holmistr reasons for missing parts is because plugins are not following specific design patterns in regards to data binding. I urge you to contact the plugin maintainers and make them aware of the issue.

https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/docs/COMPATIBILITY.md https://github.com/jenkinsci/configuration-as-code-plugin/blob/master/docs/PLUGINS.md

reitzmichnicht commented 5 years ago

Does anyone here maintain a Jenkins installation? How is this actually supposed to work? I use casc to put a configuration in the docker configuration so that the initial setup of jenkins is working with proxy, login settings ... How should this initial config be obtained from the outside? Afterwards I do not want to build a new docker image for every configuration change, so I would like to update casc from external. But this external configuration is always overridden on restart by the initial config.

jetersen commented 5 years ago

Does anyone here maintain a Jenkins installation? How is this actually supposed to work?

No reason to be snarky

I use docker to load my full config, and if further updates are required, it is done by spinning up a new container.

Though I want to switch to reading the config from URL still the full config. So that would mean I only had to build a docker container when new plugins come in, or the config URL has changed.

if I want to change the config URL it can actually be overridden by docker compose or kubernetes since the URL will only be a default value.

reitzmichnicht commented 5 years ago

Sorry don't want to reduce the great work of people behind this plugin. This is really one of the stars for jenkins in the last years. The point is its still frustrating to work with it as it is not that usable in production as I expected.

jetersen commented 5 years ago

Set your expectation accordingly and if you disagree feel free to contribute :sweat_smile:

pbsladek commented 5 years ago

@reitzmichnicht While it would be nice if this was supported.. I don't think it is a huge priority at the moment since there are a few workarounds. I'm using AWS.

# Dockerfile
ENTRYPOINT ["/bin/bash", "-c", "grab-configs.sh && /sbin/tini -- /usr/local/bin/jenkins.sh"]
#!/bin/bash -e
echo "Downloading jcasc configs"
aws s3 sync "s3://configs" "/usr/share/jenkins/ref/casc_configs"

Tested. Works great. Hopefully this helps.

riansanderson commented 5 years ago

Perhaps the best way to address this is via documentation update promoting a scripted workaround like what is proposed by @pbsladek . We run our masters on long lived VMs and I'm using puppet to manage a symlink to jenkins.yaml, but I think our use case is not typical.

Triggering a CASC load only for initial provisioning is important functionality while various plugin maintainers are (or often lack of maintainers aren't) playing catchup to fully support CASC. We use a lot (too many, but I digress) of plugins in our setup and after an instance comes up there is a certain amount of manual config that just has to happen right now before we can run our job creator.

timja commented 5 years ago

What configuration out of curiosity?

Also there's https://github.com/jenkinsci/configuration-as-code-groovy-plugin to help bridge the gap for plugins that are missing functionality currently.

riansanderson commented 5 years ago

This is a list of plugins we actively use and I configure manually:

It's better news for the ones bugs/PRs have been filed for:

I want to find the time to dive in and figure out a plugin dev setup so I can start to contribute based on the guidelines in the CASC plugin dev doc

timja commented 5 years ago

There is a PR for GHPRB: https://github.com/jenkinsci/ghprb-plugin/pull/731 Don't think that plugin is used a lot anymore, its recommended to migrate to Github branch source plugin there's a note at the top of the wiki about it: https://wiki.jenkins.io/display/JENKINS/GitHub+pull+request+builder+plugin

artifactory plugin has a github and jira issue, if you're paying them for support may we worth nudging JFrog on that

I couldn't see an issue for paramterised remote trigger may be worth creating one?

If you can write java in most cases its as simple as running mvn hpi:run from the command line or your IDE for developing plugins, sometimes you need to run mvn install -DskipTests the first time

ffissore commented 5 years ago

FYI the way I worked around this issue is to document these two post-first-run steps

An empty config will lead a no-op, so any changes made to the configuration are preserved

jetersen commented 4 years ago

@pallam-bigfoot please don't double post

pallam-bigfoot commented 4 years ago

@pallam-bigfoot please don't double post

Sorry about that. Removed my comment