Release Workflow Template

[curtisma]

Problem Statement

We have many IDS projects that require release automation so that we can quickly and easily release our projects. This will enable us to release more frequently and receive feedback more regularly.

Each IDS project has similar needs. We should be able to accommodate most projects. We should document any project structure requirements requirements

It needs to be quick and easy to add to a new IDS project. It should also be easy to make minor customizations and parameterizations for each project

Proposed Solution

We will create a template workflow that works for all IDS projects. Each project should only need to make minor customization updates. In many cases these will be limited to the ENV top-level entry

Each "Job" will have its own GitHub Action,

Steps:

• [ ] Package 1 or more conda recipes
  • [ ] Runs if a conda-recipe folder exists
  • Recipe will run the Python and skill tests
  • Recipe will run the Python and SKILL lint
  • [ ] Report the Python and SKILL Lint score
  • [ ] Add package(s) to the artifacts
• [x] Release Package
  • [ ] Release Conda package to Conda channels (dev or prod)
  • [ ] a production or dev version based on run type and input selected.
• [x] Build sphinx docs
  • [x] Created if the "Docs" folder exists
  • [x] Add docs to artifacts
• [ ] Release documentation
• [ ] Release to GitHub releases
  • See the Skywalker release workflow for an example on how to do that. It will also tag the release with the version number.
• [ ] Update Conda environments

IDS Project Structure Requirements:

• "conda-recipe" directory for the Conda recipe.
• Sphinx make file in docs/ folder and a envs/build-docs.yml Conda env definition file

Alternatives Considered

• A single workflow containing the contents of all the actions. This quickly became messy. It also doesn't allow the workflow to be automatically updated.

Proposed Plan of Action

• [ ] 1, Create the ids-actions-build-conda action
• [ ] 2. Create the ids-actions-build-sphinx-docs action
• [ ] 3. Create the ids-actions-release action
• [ ] 4. Finish the existing Softworks release script using these actions
• [ ] 5. Copy the file to the templates folder.
• [ ] 6. Try adding it to datapanels

Resources

• Run workflows locally for development: https://github.com/nektos/act
• We'll need to use context and expressions in the workflow: https://docs.github.com/en/actions/reference/context-and-expression-syntax-for-github-actions

[JadenC1126]

I can't edit this issue so I'll just comment:

for conda packages:

• use the yaml file in the project that has the same name --> i.e. for softworks it would be softworks.yml
• if that doesn't exist, use the conda env name that has the same name as the project --> i.e. softworks

ALSO possibly something to add to the IDS requirements workflow starts with "PREFIX_BUILD": "/prj/ids/ids-conda/envs/build-${{ github.event.repository.name }}"

might need to make sure we have a build-reponame environment as well ?

[JadenC1126]

Package 1 or more conda recipes

• build.yml & conda recipe needs to exist to build conda recipe

[curtisma]

This upcoming feature for reusable workflows is suppose to be ready by the end of September: https://github.com/github/roadmap/issues/98#issue-665305198

[curtisma]

Now that actions can call other actions, we should move the contents of each job into its own action.

• [ ] Conda build
• [ ] sphinx docs build
• [ ] release