search

NASA-PDS / roundup-action

Do a "roundup", a/k/a PDS-style continuous integration and delivery
Apache License 2.0
1 stars 4 forks source link

readme

🤠 PDS Engineering Actions: Roundup

This is an action for GitHub that does a "roundup"; that is, continuous integration of PDS software. (Somehow we got started on this "Western" kind of terminology and dadgum, we're stickin' with it 🤠.)

ℹ️ Using this Action

To use this action in your own workflow, just provide it with any of the following parameters:

For Maven-based roundups only, you can also specify these optional with parameters:

You'll also need this environment variable:

Depending on the roundup, you may also need the following environment variables:

🍃 Environment

The Roundup Action uses the NASA-PDS Github Actions Base as its starting environment. As of the time of this writing, this is Alpine Linux 3.16, Python 3.9.16, npm 8.10.0 and Apache Maven 3.8.5, which in turns gives you OpenJDK 17.0.10_p7 (with JAVA_HOME defaulting to /usr/lib/jvm/default-jvm).

If you need a newer or older Java, or any other packages present in order to complete the unit tests, build, documentation, etc., steps of your roundup, you can use the packages variable to specify additional Alpine Linux Packages that will be installed prior to starting. For example:

with:
    packages: openjdk11-jdk,pdfgrep

This causes the Roundup to use OpenJDK 11 and also installs the pdfgrep package.

☕️ Java Note

If you install a JDK older than OpenJDK 17.0.10_p7, you may need to also set the JAVA_HOME environment variable, as the default /usr/lib/jvm/default-jvm will point to the newest.

👮‍♂️ GitHub Admin Token

The Roundup action must have access to various target repositories. This is afforded by a token, ADMIN_GITHUB_TOKEN in the environment. Note that the NASA-PDS organization already has an ADMIN_GITHUB_TOKEN in its collection of secrets, so any repository within the organization inherits it.

But if you need to override it or set up a new token, do the following:

  1. Vist your GitHub account's Settings.
  2. Go to "Developer Settings".
  3. Go to "Personal access tokens".
  4. Press "Generate new token"; authenticate if needed.
  5. Add a note for the token, such as "PDS Ping Repo Access"
  6. Check the following scopes:
    • repo:status
    • repo_deployment
    • public_repo
  7. Press "Generate new token"

Save the token (a hex string) and install it in your source repository or organization:

  1. Visit the source repository's or organization's web page on GitHub.
  2. Go to "Settings".
  3. Go to "Secrets".
  4. Press "New secret".
  5. Name the secret, such as ADMIN_GITHUB_TOKEN, and insert the token's saved hex string as the value.
  6. Press "Add secret".

You can now (and should) destroy any saved copies of the token's hex string.

🔑 Code Signing Key

Signing code artifacts helps ensure that the code is not just created by who we say created it but that it's unmodified and free from inserted hacks like trojans or viruses. (Of course, it says nothing about the code's quality, which may be questionable or could itself be a trojan or virus.) The Roundup uses the code signing key to automatically make these assertions by signing code artifacts it sends to the OSSRH (in the future, we could also sign Python artifacts sent to the PyPI) or npmjs.com.

The NASA-PDS organization includes a CODE_SIGNING_KEY that all repositories within it inherit.

📒 Note: Whether automatically signing artifacts is a safe practice is left for future discussion! 🤔

To set up a your own code signing key for the Roundup action, first create an OpenPGP-compatible key pair using gpg or similar tool; for example, with GnuPG 2.2, run gpg --full-generate-key:

Note the hex identifier of the generated key pair; use that to export the private key and encode it as base-64, which you can then copy and paste into your repository's (or your organization's) secrets. For example, macOS users could type:

gpg --export-secret-keys HEX-KEY-ID | base64 | pbcopy

This puts the encoded private key onto your pasteboard, ready for pasting into GitHub. Use it as the value for CODE_SIGNING_KEY; see the demonstration yaml file below.

📒 Note: Don't forget to upload the corresponding public key to various keyservers, such as keys.gnupg.net, keys.openpgp.org, keyserver.ubuntu.com pool.sks-keyservers.net, etc.

🧩 Assemblies

There are several different flavors of roundups that you can specify with the assembly parameter in your workflow. The flavor of assembly tells if you're doing a stable versus an unstable software release and what steps of the roundup to perform. They are as follows:

🛫 Releases

The Roundup includes built-in support to make official releases of software, publishing artifacts to well-known repositories, and including release archives on GitHub. The PDS Java Template Repository (historically called the "generic template") and the PDS Python Template Repository (historically called the Python template) have the correct GitHub Actions workflows to support this. So does the Node.js template. If you create a new PDS repository from those templates, you're all set to roundup! Yee-haw!

To make an offical release of software version VERSION, create a tag called release/VERSION and push it to GitHub. For example, to release version 2.1.0 of your software based on the latest main:

$ git checkout main
$ git pull
$ git tag --annotate --message "Release of 2.1.0" release/2.1.0
$ git push origin release/2.1.0

If a release fails, you can retry it under some circumstances (depending on where it failed) with an invocation like:

$ git push --delete release/2.1.0
$ git push origin release/2.1.0

💁‍♀️ Demonstration

The following is a brief example how a workflow that shows how this action can be used:

name: 📦 Unstable CI/CD

on:
    push:
      branches:
          - main
        paths-ignore:
            - 'CHANGELOG.md'
            - 'docs/requirements/**'
jobs:
    unstable-roundup:
        name: 🤠 Unstable Roundup
        runs-on: ubuntu-latest
        if: github.actor != 'pdsen-ci'
        steps:
            - 
                name: 💳 Checking out repository
                uses: actions/checkout@v2
                with:
                    lfs: true
                    fetch-depth: 0
                    token: ${{secrets.ADMIN_GITHUB_TOKEN}}
            -
                name: 🐄 Rounding it up
                uses: NASA-PDS/roundup-action@main
                with:
                    assembly: unstable
                    packages: cowpoke,chili-sort,lasso
                env:
                    ADMIN_GITHUB_TOKEN: ${{secrets.pat}}
                    CODE_SIGNING_KEY:   ${{secrets.CODE_SIGNING_KEY}}
                    ossrh_username:     jocowboy
                    ossrh_password:     ${{secrets.OSSRH_PASSWORD}}
                    pypi_username:      snakewrangler
                    pypi_password:      ${{secrets.PYPI_PASSWORD}}

🔧 Development

Make a local image for testing:

docker image build --tag pds-roundup:latest .

You can then poke around in it:

docker container run --interactive --tty --rm --name roundup --volume ${PWD}:/mnt --entrypoint /bin/sh pds-roundup:latest

But you could also invoke it the way GitHub Actions does; for example:

docker container run --interactive --tty --rm --name roundup-dev --workdir /github/workspace \
    --env INPUT_MODE --env HOME --env GITHUB_JOB --env GITHUB_REF --env GITHUB_SHA --env GITHUB_REPOSITORY \
    --env GITHUB_REPOSITORY_OWNER --env GITHUB_RUN_ID --env GITHUB_RUN_NUMBER --env GITHUB_ACTOR \
    --env GITHUB_WORKFLOW --env GITHUB_HEAD_REF --env GITHUB_BASE_REF --env GITHUB_EVENT_NAME \
    --env GITHUB_SERVER_URL --env GITHUB_API_URL --env GITHUB_GRAPHQL_URL \
    --env GITHUB_ACTION --env GITHUB_EVENT_PATH --env GITHUB_PATH --env GITHUB_ENV --env RUNNER_OS \
    --env RUNNER_TOOL_CACHE --env RUNNER_TEMP --env RUNNER_WORKSPACE --env ACTIONS_RUNTIME_URL \
    --env ACTIONS_RUNTIME_TOKEN --env ACTIONS_CACHE_URL --env GITHUB_ACTIONS=true --env CI=true \
    --env GITHUB_WORKSPACE=/github/workspace \
    --env GITHUB_TOKEN=$(cat my-dev-token.txt) \
    --env ADMIN_GITHUB_TOKEN=$(cat my-dev-token.txt) \
    --env pypi_username=joe_cowboy4life \
    --env pypi_password=s3cr3t \
    --env ossrh_username=java_cowboy4life \
    --env ossrh_password=m0rec0ff33 \
    --env GITHUB_REPOSITORY=joecowboy/test-repo \
    --volume /Users/joe/Documents/Development/test-repo:"/github/workspace" \
    pds-roundup --debug --assembly unstable

Or build it to run outside of a container; for example:

python3 -m venv venv
venv/bin/pip install --quiet --upgrade setuptools wheel pip
venv/bin/pip install --editable .

Then add venv/bin/pip to your PATH and use the convenience script, support/run-roundup.sh:

cd my-project-directory
../roundup-action/support/run-roundup.sh unstable

Note that to use run-roundup.sh you need to set quite a few other environment variables, prepare some files, and ensure certain commands are on the PATH. See the comments in the script for details.