Open oleg-nenashev opened 4 years ago
@timja @jonbrohauge This is what we were talking about on Oct at the end of the meeting
OKay, I have finally forgot about it. Sorry all. Once I get back to JCasC, getting the patch over the line will be my top priority
Any update on this issue?
Nope. I was unable to finish it due to COVID-19 and other emergencies. No ETA, sorry
On Wed, Jul 1, 2020, 11:22 qalinn notifications@github.com wrote:
Any updated on this issue?
— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/jenkinsci/configuration-as-code-plugin/issues/1141#issuecomment-652302685, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAW4RIFZK2DYJR72URLS3NDRZL56LANCNFSM4I7MQIMQ .
Better late than never, working on it again. Thanks to recent patches by @jetersen , now we have a common way to extend variable resolution methods.
I like the idea. Looks similar to what eyaml
provides for Puppet's Hiera (looks like ENC[PKCS7, encrypted text]
there). But eyaml
in itself is just a framework which can be enhanced with different encryption backends (like Vault, GnuPG, KMS,...). We're using it with the GnuPG backend (ENC[GPG, encrypted text]
) since it offers the most flexibility. Encrypting the secrets with multiple keys at once allows them to be decrypted/edited by multiple users, using their own keys, and also to reuse them on several installations, which can also have their own keypairs each. eyaml
also comes with a handy command line tool which allows for easy re-encryption in case of key changes.
Would be nice to have similar functionality available here, too.
Hi
Until it's done, can you give pointers to the alternative methods ? I'm looking for a way to migrate secrets from an old to a new Jenkins. A one time operation. Can I force Jenkins' internal secret key for example ?
Thanks,
Hi
Until it's done, can you give pointers to the alternative methods ? I'm looking for a way to migrate secrets from an old to a new Jenkins. A one time operation. Can I force Jenkins' internal secret key for example ?
Thanks,
Yes, alternatives would be appreciated.
We came up with a couple of groovy script to export credentials in the JCasC format. Here they are, for anyone interested.
Note, they fit our use case with our Username & Password + String + File + SSH credentials, if you've additional kind of credentials you'll need to add stuff.
def creds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials()
def credsFile = new File('/tmp/secrets/all-secrets.yaml')
for(c in creds) {
if(c instanceof com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey){
yaml = String.format(
'''\
- basicSSHUserPrivateKey:
scope: "GLOBAL"
id: "%s"
description: "%s"
username: "%s"
privateKeySource:
directEntry:
privateKey: "%s"
''',
c.id,
c.description,
c.username,
c.privateKeySource.getPrivateKeys()[0],
)
print(yaml)
credsFile.append(yaml)
}
if (c instanceof com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl){
yaml = String.format(
'''\
- usernamePassword:
scope: "GLOBAL"
id: "%s"
description: "%s"
username: "%s"
password: "%s"
''',
c.id,
c.description,
c.username,
c.password,
)
print(yaml)
credsFile.append(yaml)
}
if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl){
yaml = String.format(
'''\
- string:
scope: "GLOBAL"
id: "%s"
description: "%s"
secret: "%s"
''',
c.id,
c.description,
c.secret,
)
print(yaml)
credsFile.append(yaml)
}
if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){
yaml = String.format(
'''\
- file:
scope: "GLOBAL"
id: "%s"
description: "%s"
fileName: "%s"
secretBytes: "%s"
''',
c.id,
c.description,
c.fileName,
c.secretBytes.plainData.encodeBase64(),
)
print(yaml)
credsFile.append(yaml)
}
}
And we needed a 2nd one for a sub cred domain "Debian package builder"
def domainCreds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getDomainCredentials()
for (domainCred in domainCreds) {
if (domainCred.domain.name != "Debian package builder") {
continue
}
def credsFile = new File('/tmp/secrets/builder.yaml')
for (c in domainCred.getCredentials()) {
if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){
yaml = String.format(
'''\
- file:
scope: "GLOBAL"
id: "%s"
description: "%s"
fileName: "%s"
secretBytes: "%s"
''',
c.id,
c.description,
c.fileName,
c.secretBytes.plainData.encodeBase64(),
)
print(yaml)
credsFile.append(yaml)
}
}
Hi, is there an Update for an ETA? Cheers, Philipp
Hi , would be nice to have that feature.. cheers Michael
Since JCasC can use environment variables to fill in credentials, we've solved this by switching to Vault as an external credentials provider (needs Vault plugin). If setup properly, JCasC can then read its initial (Vault approle) credentials from Vault itself (yes, sounds weird), treated as environment variables. The setup looks like this (assuming your Jenkins version already comes with systemd service file):
approle
login provider and kv2
secrets storesecret/jenkins/jcasc
in the k/v store containing two k/v pairs: [VAULT_ROLE_ID, /var/lib/jenkins/vault
with the following content (make sure to protect it properly):
CASC_VAULT_APPROLE=<your role id>
CASC_VAULT_APPROLE_SECRET=<your secret id>
CASC_VAULT_PATHS=secret/jenkins/jcasc
CASC_VAULT_URL=<vault url>
/etc/systemd/system/jenkins.service.d/
(name doesn't matter, but must have a .conf
extension), with the following content:
[Service]
Environment="CASC_VAULT_FILE=/var/lib/jenkins/vault"
systemctl daemon-reload
and systemctl restart jenkins.service
credentials:
section (the variables should match the k/v pairs from step 3 above):
credentials:
system:
domainCredentials:
- credentials:
- vaultAppRoleCredential:
description: "Jenkins credentials for accessing Vault"
id: "JenkinsApprole"
path: "approle"
roleId: "${VAULT_ROLE_ID}"
scope: SYSTEM
secretId: "${VAULT_SECRET_ID}"
- vaultUsernamePasswordCredentialImpl:
description: "Some User"
engineVersion: 2
id: "SOME_USER"
passwordKey: "password"
path: "secret/jenkins/some_user"
scope: GLOBAL
usernameKey: "username"
- vaultSSHUserPrivateKeyImpl:
description: "Some User (SSH)"
engineVersion: 2
id: "SOME_USER_SSH"
passphraseKey: "key_passphrase"
path: "secret/jenkins/some_user"
privateKeyKey: "private_key"
scope: GLOBAL
NOTE: The Vault plugin doesn't support all credential types, yet (AWS for example) and there are also some plugins which don't use the Jenkins credential system at all. In this case you can still work around this by adding more variables to secret/jenkins/jcasc
and reference those just as we did in step 7.
We came up with a couple of groovy script to export credentials in the JCasC format. Here they are, for anyone interested.
Note, they fit our use case with our Username & Password + String + File + SSH credentials, if you've additional kind of credentials you'll need to add stuff.
def creds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getCredentials() def credsFile = new File('/tmp/secrets/all-secrets.yaml') for(c in creds) { if(c instanceof com.cloudbees.jenkins.plugins.sshcredentials.impl.BasicSSHUserPrivateKey){ yaml = String.format( '''\ - basicSSHUserPrivateKey: scope: "GLOBAL" id: "%s" description: "%s" username: "%s" privateKeySource: directEntry: privateKey: "%s" ''', c.id, c.description, c.username, c.privateKeySource.getPrivateKeys()[0], ) print(yaml) credsFile.append(yaml) } if (c instanceof com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl){ yaml = String.format( '''\ - usernamePassword: scope: "GLOBAL" id: "%s" description: "%s" username: "%s" password: "%s" ''', c.id, c.description, c.username, c.password, ) print(yaml) credsFile.append(yaml) } if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl){ yaml = String.format( '''\ - string: scope: "GLOBAL" id: "%s" description: "%s" secret: "%s" ''', c.id, c.description, c.secret, ) print(yaml) credsFile.append(yaml) } if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){ yaml = String.format( '''\ - file: scope: "GLOBAL" id: "%s" description: "%s" fileName: "%s" secretBytes: "%s" ''', c.id, c.description, c.fileName, c.secretBytes.plainData.encodeBase64(), ) print(yaml) credsFile.append(yaml) } }
And we needed a 2nd one for a sub cred domain "Debian package builder"
def domainCreds = com.cloudbees.plugins.credentials.SystemCredentialsProvider.getInstance().getDomainCredentials() for (domainCred in domainCreds) { if (domainCred.domain.name != "Debian package builder") { continue } def credsFile = new File('/tmp/secrets/builder.yaml') for (c in domainCred.getCredentials()) { if (c instanceof org.jenkinsci.plugins.plaincredentials.impl.FileCredentialsImpl){ yaml = String.format( '''\ - file: scope: "GLOBAL" id: "%s" description: "%s" fileName: "%s" secretBytes: "%s" ''', c.id, c.description, c.fileName, c.secretBytes.plainData.encodeBase64(), ) print(yaml) credsFile.append(yaml) } }
Hi, would you please add details on how to use this on the end side (import)? I guess you use docker swarm?
hi, is there any ETA for this?
Hi, I have the same issue, some news?
No one is actively working on this.
As a user I want to share a single configuration file between multiple Jenkins instance, including credential definitions. Currently JCasC support plugin supports defining encrypted secrets on the configuration YAML. Configuration example:
Encryption is done using the Jenkins-internal secret key which is unique for every Jenkins instance. It means that the credentials are not portable between instances. It also creates obstacles for immutable images which start with a fresh Jenkins instance and initially do not have an initialized secret key for encryption. Although there are workarounds, I suggest adding support of external certificates.
Proposal:
{ENC, PKCS7,AQAAABAAAAAQ1/JHKggxIlBcuVqegoa2AdyVaNvjWIFk430/vI4jEBM=}
(encryptted text)Implementation notes:
SecretSource
class which includes underlying extensions for encryption methods