Reducing security risk in our GitHub workflows

[consideRatio]

I've created this issue to coordinate security work across the org.

As initiated by @manics practically in https://github.com/jupyterhub/action-major-minor-tag-calculator/pull/75 and by the discussion in https://github.com/jupyterhub/team-compass/issues/398#issuecomment-823825014, we should do various things for our GitHub repos' GitHub workflows to improve the security posture against compromised GitHub actions.

Action points for each repo

In all GitHub workflows' jobs that are referencing a GitHub secret such as secrets.GITHUB_TOKEN or a manually configured such as secrets.DOCKERHUB_TOKEN, we should:

• Consider what actions we use and if we trust them

• Pin the action references to a specific git commit hashes and make an inline comment # associated tag: v1.2.3 for easier maintainability

  # acceptable - github official actions
  - uses: actions/checkout@v2
  - uses: actions/setup-python@v2
  - uses: actions/setup-node@v1
  - users: actions/upload-artifact@v2
  
  # correct
  - name: Set up QEMU (for docker buildx)
  uses: docker/setup-qemu-action@25f0500ff22e406f7191a2a8ba8cda16901ca018 # associated tag: v1.0.2
  
  # incorrect (but acceptable in job's not using any GitHub secrets)
  - name: Set up QEMU (for docker buildx)
  uses: docker/setup-qemu-action@v1

• If needed, we should add write permissions explicitly for the secrets.GITHUB_TOKEN which previously had them by default but doesn't any more as we made them read-only by default in the JupyterHub github org. This will be needed for jobs that update branches or tags in the repo for example.

  jobs:
  my-job:
    permissions:
      contents: write

Non-archived repositories to consider

Has GHA in use

• [x] jupyterhub/action-k3s-helm (https://github.com/jupyterhub/action-k3s-helm/pull/28)
• [x] jupyterhub/action-k8s-await-workloads (https://github.com/jupyterhub/action-k8s-await-workloads/pull/30)
• [x] jupyterhub/action-k8s-namespace-report (https://github.com/jupyterhub/action-k8s-namespace-report/pull/8)
• [x] jupyterhub/action-major-minor-tag-calculator (https://github.com/jupyterhub/action-major-minor-tag-calculator/pull/75)
• [x] jupyterhub/batchspawner (OK as it is)
• [x] jupyterhub/binder (OK as it is) Hasn't started using GHA for documentation publishing.
• [x] jupyterhub/binderhub (https://github.com/jupyterhub/binderhub/pull/1292)
• [x] jupyterhub/chartpress (OK as it is)
• [x] jupyterhub/configurable-http-proxy (https://github.com/jupyterhub/configurable-http-proxy/pull/307)
• [ ] jupyterhub/dockerspawner Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/jupyter-remote-desktop-proxy Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/jupyter-server-proxy Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/jupyterhub-idle-culler Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [x] jupyterhub/jupyterhub (https://github.com/jupyterhub/jupyterhub/pull/3436)
• [ ] jupyterhub/kubespawner Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/ldapauthenticator Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/ltiauthenticator Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/mybinder.org-deploy Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824742867
• [ ] jupyterhub/nativeauthenticator Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511 Missing credentials to push to PyPI (https://github.com/jupyterhub/nativeauthenticator/issues/144)
• [ ] jupyterhub/nbgitpuller Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/oauthenticator Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [x] jupyterhub/pebble-helm-chart (OK as it is)
• [ ] jupyterhub/repo2docker-action I think there is work that needs to be done here with regards to being explicit about the need for GITHUB_TOKEN's permissions. I created https://github.com/jupyterhub/repo2docker-action/issues/64 to represent this.
• [ ] jupyterhub/repo2docker Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [ ] jupyterhub/sudospawner No publish workflow setup: https://github.com/jupyterhub/sudospawner/issues/68
• [x] jupyterhub/the-littlest-jupyterhub (OK as it is)
• [ ] jupyterhub/traefik-proxy Discussed: https://github.com/jupyterhub/team-compass/issues/404#issuecomment-824714511
• [x] jupyterhub/wrapspawner (OK as it is)
• [x] jupyterhub/zero-to-jupyterhub-k8s (https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pull/2163)

Doesn't have GHA in use, but have a need for tests/GHA

• [ ] jupyterhub/autodoc-traits
• [ ] jupyterhub/firstuseauthenticator
• [ ] jupyterhub/hubshare
• [ ] jupyterhub/jupyter-rsession-proxy
• [ ] jupyterhub/kerberosauthenticator
• [ ] jupyterhub/nullauthenticator
• [ ] jupyterhub/simplespawner
• [ ] jupyterhub/systemdspawner
• [ ] jupyterhub/tmpauthenticator
• [ ] jupyterhub/yarnspawner

Doesn't have GHA in use, and have no need for it

• jupyterhub/.github
• jupyterhub/binder-billing
• jupyterhub/binder-data
• jupyterhub/design
• jupyterhub/helm-chart
• jupyterhub/jupyterhub-deploy-docker
• jupyterhub/jupyterhub-deploy-hpc
• jupyterhub/jupyterhub-deploy-teaching
• jupyterhub/jupyterhub-example-kerberos
• jupyterhub/jupyterhub-on-hadoop
• jupyterhub/jupyterhub-the-hard-way
• jupyterhub/jupyterhub-tutorial
• jupyterhub/katacoda-scenarios
• jupyterhub/mybinder-tools
• jupyterhub/outreachy
• jupyterhub/team-compass

Script to list repos of relevance

I used this script to get the names of repo's of relevance to create the lists above.

curl -H "Accept: application/vnd.github.v3+json"  "https://api.github.com/orgs/jupyterhub/repos?per_page=100" > tmp.json
cat tmp.json | jq '.[] | select(.archived == false) | .full_name'

[minrk]

More information in the github docs but there's also a degree of trust that informs how to do each pin:

1. pinning sha means we don't need to trust the action repo (or github itself) to protect against malicious commits
2. pinning tag means we trust the repo maintainer to protect their tags

The downside of 1. is that we need to update actions often, and especially means we won't get security updates if we don't stay on top of it (dependabot helps, I think?). So there can actually be a downside security-wise of pinning if it's the action itself that has a vulnerability (e.g. the codecov action contents had the vulnerability, the action repo itself was not compromised). For instance, I think it would be reasonable to trust @docker as much as @actions, but we can make that choice on a per-org or even per-action basis.

[consideRatio]

@minrk good points, then I suggest we trust @docker alongside @actions going onwards and raise discussions if there are other orgs and associated actions we want to trust or not.

[consideRatio]

dockerspawner, jupyter-remote-desktop-proxy, jupyter-server-proxy, jupyterhub-idle-culler: pypa/gh-action-pypi-publish@v1.4.1 / @v1.3.0 (latest is 1.4.2)

Feels okay for me to trust even though I would prefer we just do the terminal command ourselves directly to upload a package. Perhaps use @release/v1 which is a branch with the latest v1 releases instead of @v1.4.1 if we use the action?

Suggestion: we retain use of the pypa action, we trust it, and reference @release/v1 branch. Ok?

[consideRatio]

@sgibson91 @betatim @minrk what do you think about use of the following actions in mybinder.org-deploy's CD workflow jobs?

• I think this use of a secret in a pull_request triggered workflow is harmless because it won't be made available. But, if it won't have any effect, we should not have it there just to be sure.

• We use: google-github-actions/setup-gcloud@master (v0 is available, v0.2.1 is latest), azure/setup-kubectl@v1, sliteteam/github-action-git-crypt-unlock@a09ea5079c1b0e1887d4c8d7a4b20f00b5c2d06b, azure/docker-login@v1, nick-invision/retry@39da88d5f7d15a96aed861dbabbe8b7443e3182a

  I think we pin where it make sense etc for these actions.

[minrk]

Suggestion: we retain use of the pypa action, we trust it, and reference @release/v1 branch. Ok?

Yeah, I think that's okay. It doesn't do much, so we could also do our own python -m build . && twine upload dist/* which works for ~all Python packages like you did for docker login.

[consideRatio]

Yeah, I think that's okay. It doesn't do much, so we could also do our own python -m build . && twine upload dist/* which works for ~all Python packages like you did for docker login.

In https://github.com/jupyterhub/sudospawner/pull/69/files#diff-87db21a973eed4fef5f32b267aa60fcee5cbdf03c67fafdc2a9b553bb0b15f34R23-R36 I've added a suggestion on the equivalent of using the PyPA action.

It is pretty much to run: twine check and twine upload, of which twine check now can be part of our test.

[consideRatio]

I don't think we should track this further than we have done by making the default permission for GITHUB_TOKEN that actions can access be contents:read in the entire jupyterhub org.