search

pangeo-data / pangeo-docker-images

Docker Images For Pangeo Jupyter Environment
https://pangeo-docker-images.readthedocs.io
MIT License
125 stars 90 forks source link

Simplify condalock workflow to work without chatops #414

Open scottyhq opened 1 year ago

scottyhq commented 1 year ago

Discussed in detail here https://github.com/CryoInTheCloud/hub-image/issues/18

A less convoluted workflow for creating and installing from a unified lockfile from conda-lock would be nice.

The chatops /condalock command was a fun solution at the time to allow administrators to trigger the condalock workflow (that requires a token with write permissions to add files to a PR), but there are alternatives now such as manually triggered workflow_dispatch or adding a label to an open PR https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

weiji14 commented 1 year ago

Hi @scottyhq, I've been discussing with @yuvipanda and a few others on turning the /condalock chatops into a GitHub Action or GitHub App in a dedicated repo under the jupyter org (though we'll probably test things on CryoInTheCloud or some other org first). There's been some interest from 2i2c and the folks at NASA IMPACT/VEDA to put some effort into this, and we can rope you into the discussion if you're interested!

The chatops /condalock command was a fun solution at the time to allow administrators to trigger the condalock workflow (that requires a token with write permissions to add files to a PR), but there are alternatives now such as manually triggered workflow_dispatch or adding a label to an open PR https://securitylab.github.com/research/github-actions-preventing-pwn-requests/

On this conda-lock trigger mechanism, could you elaborate on what's the ideal way to set things up from a repo maintainer perspective? Currently the /condalock chatops is a bit hacky, though it allows just about any GitHub user to trigger the locking mechanism. Were you thinking of a more secure way of triggering the workflow that requires approval on first run (like most GitHub Actions) instead?

cisaacstern commented 1 year ago

In case it's useful, here's a snippet from a workflow over in pangeo-forge-runner that triggers an integration test based on the presence of a particular label:

https://github.com/pangeo-forge/pangeo-forge-runner/blob/3b71752c3654b3fbab39ad242ba7901b166902ac/.github/workflows/dataflow.yaml#L1-L24

The workflow triggers on every commit to a PR (if the specified label is also present at commit time). It also triggers against the latest commit, if the label is added.

yuvipanda commented 1 year ago

I do think just typing /condalock is perfectly fine, and the way to automate it is perhaps to have another bot that does /condalock whenever an update is done - or even be automatically triggered each time a push is made (similar to how pre-commit.ci does it). It's a fairly harmless action (unlike pangeo-forge runs!) that we can hand out broader control to the world to

scottyhq commented 1 year ago

There's been some interest from 2i2c and the folks at NASA IMPACT/VEDA to put some effort into this, and we can rope you into the discussion if you're interested

Cool! Thanks @weiji14 @cisaacstern @yuvipanda for the ideas. I'm definitely interested but don't really have time to work on it. Any simplifications to the current setup or ways to make it more easily setup in other repositories would be fantastic.

weiji14 commented 1 year ago

There's been some interest from 2i2c and the folks at NASA IMPACT/VEDA to put some effort into this, and we can rope you into the discussion if you're interested

Cool! Thanks @weiji14 @cisaacstern @yuvipanda for the ideas. I'm definitely interested but don't really have time to work on it. Any simplifications to the current setup or ways to make it more easily setup in other repositories would be fantastic.

Totally understand, I'll keep you in the loop on the outcome when we start implementing this :smile:. If it's ok, I might open a few PRs on pangeo-docker-images to test things out in the next few months. Have recently got conda-lock=2.0 to work on https://github.com/CryoInTheCloud/hub-image/pull/14, and would be nice to roll that out to pangeo-docker-images soon too!

weiji14 commented 1 year ago

Started initial implementation of a composite GitHub Action at https://github.com/weiji14/conda-lock-refresh, with related demo repo at https://github.com/weiji14/conda-lock-refresh-demo. Note that I'm still testing this out, and there's still a few bugs (~e.g. the 'bot' pushes to the main branch instead of the Pull Request branch, gets triggered on regular comments without /condalock, etc~ Edit: those two bugs have been fixed, see https://github.com/weiji14/conda-lock-refresh-demo/pull/6#issuecomment-1691100735), but it's looking promising :slightly_smiling_face:

Once this work is a bit more polished, I can transfer ownership of these repos to a more official organization.