search

riboseinc / rpm-specs

Ribose RPM specs
1 stars 1 forks source link

Automatic signing and publishing to riboseinc:yum #37

Closed dewyatt closed 5 years ago

dewyatt commented 5 years ago

As discussed in https://github.com/riboseinc/rpm-spec-botan2/issues/3

ronaldtse commented 5 years ago

@dewyatt is there anything I need to do here? Thanks!

dewyatt commented 5 years ago

@ronaldtse Yes, please:

  1. Optional, but may make things easier. Create a public subgroup on gitlab for rpm-specs* and move all those repos into that group. Settings -> General -> Advanced -> Transfer project for each repo.
  2. Set the GITHUB_API_TOKEN protected variable for creating the github releases. If using the subgroup, just go to that group and set it under Settings -> CI/CD -> Environment variables (and make sure you mark it as protected). If not using a subgroup, it will have to be done for each rpm-spec-* repo individually.
  3. As above, can you create variables to hold the signing information? PACKAGE_SIGNING_KEY (armored private key), PACKAGE_SIGNING_NAME (userid).

(variable naming at your discretion, just using examples here)

ronaldtse commented 5 years ago

@dewyatt thanks for the detailed instructions! I just spoke with @ribose-jeffreylau and he will handle this. He'll keep you updated!

ribose-jeffreylau commented 5 years ago

Hi @dewyatt , I created the ribose/rpm-spec sub-group and moved all rpm-spec* projects over.

As for the package signing key, I created PACKAGE_SIGNING_KEY_BASE64 which is a base64-encoded string of the private PGP signing key, devoid of any whitespace to avoid shell issues (We had issues with that in Jenkins). We have been using something like the following in our Jenkins packaging jobs:

# Prepare Yum package key
echo "${PACKAGE_SIGNING_KEY_BASE64}" | ruby -r base64 -e 'puts Base64.decode64 ARGF.read' > "${PACKAGER_KEY_PATH}"

... where PACKAGER_KEY_PATH can be somewhere temporary like /tmp/packager.key (This also has been set in GitLab).

I'm not sure if GitLab handles whitespaces and newlines in variables well enough. So we may or may not need the above snippet in the GitLab CI config.

As for PACKAGE_SIGNING_NAME, I don't think we had that, but I believe something like the following could achieve the same thing:

export PACKAGE_SIGNING_NAME="$(gpg --list-packets "${PACKAGER_KEY_PATH}"  | grep user.ID | sed -Ee 's/^.*packet: "(.*)"$/\1/')"

Let us know if there's anything missing. Cheers!

ribose-jeffreylau commented 5 years ago

I also turned off Public pipelines from the CI/CD config for all rpm-spec* projects, if that would help preventing leak.

EDIT: This also had to be done individually on each project setting page -- I don't know of a way to enforce this group-wide. As an aside, I mass-updated the settings with the help of this GitLab Ruby gem.

dewyatt commented 5 years ago

Thanks @ribose-jeffreylau ! This is unrelated to this particular issue, but I'd also really like to complete the GitHub Releases item on my TODO list:

ronaldtse commented 5 years ago

Thanks guys, @dewyatt I'm sure @ribose-jeffreylau will get back to you as soon as he could.

ribose-jeffreylau commented 5 years ago

Hi @dewyatt ,

ronaldtse commented 5 years ago

@ribose-jeffreylau @dewyatt I believe we can use the GitHub Deploy Token to perform the RPM push to https://github.com/riboseinc/yum . This way we don't need to pretend to be another user and have better security.

dewyatt commented 5 years ago

@ronaldtse Do you mean using a deploy key as you did in ogc-tcpnp?

I was hoping to go that route as well, I just haven't made it to the point of asking for it yet (want to finish package signing first).

Anyways, just as a summary of the state of things:

Automated Task Status Permissions Method
GitHub Releases Completed Write access to rpm-spec-* repos. GITHUB_API_TOKEN protected env var.
Package Signing Completed Access to a signing key. PACKAGE_SIGNING_KEY_BASE64 protected env var.
riboseinc/yum Repo Updates Pending Push access to the yum repo. Access to the signing key to sign repo metadata. Suggest a deploy key for the riboseinc/yum repo.
SRPMs Pending N/A N/A

Ideas for improvement:

  1. Restrict the GITHUB_API_TOKEN to only have write access to the rpm-spec-* repos, rather than all public repos. I know Jeffrey mentioned he is already thinking about improving this.
  2. We could use a deploy key for the GitHub Releases, which would be a bit more secure, but this requires creating a key for each repo and setting an env var for each repo, which is a lot of overhead.
dewyatt commented 5 years ago

Package signing should be complete, tested here: https://github.com/riboseinc/rpm-spec-libgsf/releases/tag/1.14.45-1.el7

I successfully verified with the 2D8998FA key from the riboseinc/yum repo.

On a side note, I just realized I should probably add SRPMs to the automation TODO list.

ronaldtse commented 5 years ago

@dewyatt Yes, by a "deploy key" I meant the per-repo deploy key that can update a GitHub repository (e.g. https://github.com/riboseinc/ogc-tcpnp/settings/keys). Since our riboseinc/yum repo only needs to have content pushed to the git repo, we could use a deploy key there.

  1. We could use a deploy key for the GitHub Releases, which would be a bit more secure, but this requires creating a key for each repo and setting an env var for each repo, which is a lot of overhead.

The drawback of a deploy key is that it cannot be used to create/push GitHub Releases, so we still need the GITHUB_API_TOKEN (of a user).

On a side note, I just realized I should probably add SRPMs to the automation TODO list.

Yes let's do that 👍

dewyatt commented 5 years ago

@ronaldtse or @ribose-jeffreylau when you get a chance, can someone store the yum repo deploy key in a protected env var for the rpm-spec group (maybe YUM_DEPLOY_KEY_BASE64)?

ribose-jeffreylau commented 5 years ago

@dewyatt I'm on it.

ribose-jeffreylau commented 5 years ago

@dewyatt Hi, you can now find it in GITHUB_YUM_DEPLOY_KEY_BASE64.

cc: @kwkwan