Support pre and post steps in Composite Actions

[umarcor]

Describe the enhancement

Currently, three Action types are supported: Docker, JavaScript and Composite (see https://docs.github.com/en/actions/creating-actions/about-custom-actions#types-of-actions). However, features pre, pref-if, post and post-if are only is supported in JavaScript Actions only (see https://docs.github.com/en/actions/creating-actions/metadata-syntax-for-github-actions#runs-for-javascript-actions). Therefore, users writing workflows/actions using some scripting language such as Python are forced to wrap the steps in JavaScript in order to register pre and post steps. See, for example, https://github.com/pyTooling/Actions/tree/main/with-post-step:

name: With post step
description: 'Generic JS Action to execute a main command and set a command in a post step.'
inputs:
  main:
    description: 'Main command/script.'
    required: true
  post:
    description: 'Post command/script.'
    required: true
  key:
    description: 'Name of the state variable used to detect the post step.'
    required: false
    default: POST
runs:
  using: 'node12'
  main: 'main.js'
  post: 'main.js'

const { exec } = require('child_process');

function run(cmd) {
  exec(cmd, (error, stdout, stderr) => {
    if ( stdout.length != 0 ) { console.log(`${stdout}`); }
    if ( stderr.length != 0 ) { console.error(`${stderr}`); }
    if (error) {
      process.exitCode = error.code;
      console.error(`${error}`);
    }
  });
}

const key = process.env.INPUT_KEY.toUpperCase();

if ( process.env[`STATE_${key}`] != undefined ) { // Are we in the 'post' step?
  run(process.env.INPUT_POST);
} else { // Otherwise, this is the main step
  console.log(`::save-state name=${key}::true`);
  run(process.env.INPUT_MAIN);
}

The complexity might be simplified if Python or Bash or PowerShell were supported similarly to JavaScript:

name: 'Login to container registries and set a post step'

runs:
  using: 'python'
  main: 'precmd.py'
  post: 'postcmd.py'

Additional information

Alternatively, since regular steps support Python as a built-in shell already, the same capability might be achieved if Composite Actions supported fields pre and post as a complement to steps. For instance:

name: 'Login to container registries and set a post step'

inputs:
  precmd:
    description: 'Pre command'
    required: true
  postcmd:
    description: 'Pre command'
    required: true

runs:
  using: 'composite'
  pre:
    - shell: python
      run: ${{ inputs.precmd }}
  post:
    - shell: python
      run: ${{ inputs.postcmd }}
  steps:
    - ...

/cc @thboop, per https://github.com/actions/runner/issues/646#issuecomment-901336347

[webknjaz]

I've also faced this need. Subscribing.

[umarcor]

@webknjaz you might want to use hdl/containers/utils/with-post-step, or just copy the sources to you own repo: https://github.com/pyTooling/Actions/tree/main/with-post-step. For instance:

  - name: Release
    uses: ./utils/with-post-step
    with:
      main: |
        echo '${{ inputs.gcr_token }}' | docker login gcr.io -u _json_key --password-stdin
        echo '${{ inputs.gh_token }}' | docker login ghcr.io -u gha --password-stdin
        echo '${{ inputs.docker_pass }}' | docker login docker.io -u '${{ inputs.docker_user }}' --password-stdin
        dockerRelease ${{ inputs.architecture }} ${{ inputs.collection }} ${{ inputs.images }}
      post: for registry in gcr.io ghcr.io docker.io; do docker logout "$registry"; done

[webknjaz]

@umarcor I was considering something like this. Will it run at the end of the job execution or at the composite action exit?

[umarcor]

When I use it in a composite action, I call that action once only. Therefore, there is no difference between the end of the composite action or the end of the job. However, according to documentation, all post steps are executed at the end of the job, in reverse order.

[ethomson]

Supporting pre and post in composite actions seems like a usable addition. We don't have this on our roadmap at the moment, but I'll add it to the list for future work. Thanks for the suggestion!

[jessehouwing]

I have a use-case for this as well having a runs-post: | block would also work for me, that's basically the trick these custom post-step actions employ. But then you get to pick the kind of script host and add environment variables and such.

In this case I want to inject a task at the end of the workflow that keeps the runner alive a bit longer for debugging purposes.

Adding a wait at the end works as a workaround for now:

- run: |
        Start-Sleep -seconds 300
      shell: pwsh

[jessehouwing]

@webknjaz you might want to use hdl/containers/utils/with-post-step, or just copy the sources to you own repo: https://github.com/pyTooling/Actions/tree/main/with-post-step. For instance:

  - name: Release
    uses: ./utils/with-post-step
    with:
      main: |
        echo '${{ inputs.gcr_token }}' | docker login gcr.io -u _json_key --password-stdin
        echo '${{ inputs.gh_token }}' | docker login ghcr.io -u gha --password-stdin
        echo '${{ inputs.docker_pass }}' | docker login docker.io -u '${{ inputs.docker_user }}' --password-stdin
        dockerRelease ${{ inputs.architecture }} ${{ inputs.collection }} ${{ inputs.images }}
      post: for registry in gcr.io ghcr.io docker.io; do docker logout "$registry"; done

Anyone know what he magic syntax would be to include such local action in the composite action repo?

    - name: Wait for user to terminate workflow
      uses: ./with-post-step
      with:

in the action YAML will be relative to therepo root, not relative to the composite's action.yaml file.

[umarcor]

Maybe uses: ${{ github.action_path }}/with-post-step? (I did not try it)

[jessehouwing]

Tried that. Not making actions happy:

Error: jessehouwing/debug-via-ssh/main/action.yaml (Line: 154, Col: 13):
Error: jessehouwing/debug-via-ssh/main/action.yaml (Line: 154, Col: 13): Unrecognized named-value: 'github'. Located at position 1 within expression: github.action_path
Error: jessehouwing/debug-via-ssh/main/action.yaml (Line: 154, Col: 13): Expected format {org}/{repo}[/path]@ref. Actual '${{ github.action_path }}/with-post-step'
Error: System.FormatException: Input string was not in a correct format.

For now fixed by passing in the full action path:

uses: jessehouwing/debug-via-ssh/with-post-step@main

But I consider this a bug... I'd expect a composite action to be able to reference its own local actions.

[umarcor]

@jessehouwing see https://github.com/github/feedback/discussions/9049 (via https://github.com/hdl/containers/issues/48).

[Kurt-von-Laven]

This would make it a lot easier to create quick and easy actions that cache things for various common tools/languages. Excited to hear it's already on the roadmap!

[menasheh]

When is this planned? Seems like a fundamental feature to me

[harzallah]

+1

[noahsbwilliams]

Just writing in support - would have a lot of value in the context of "write secret --> $action --> remove secret" tasks

[StephenHodgson]

Please add this functionality, thanks

[fabasoad]

Supporting pre and post in composite actions seems like a usable addition. We don't have this on our roadmap at the moment, but I'll add it to the list for future work. Thanks for the suggestion!

Hi @ethomson! Do you have any news for us? Had it been added to the roadmap (if yes, maybe you could share the ETA) or not yet? Thanks.

[umarcor]

@fabasoad, according to the bio (https://github.com/ethomson) he does not work for GitHub anymore. You might want to ping/ask @chrispat and/or @TingluoHuang.

[scottgigante-immunai]

+1 for this being a useful feature!

[philomory]

Rather that simply emphasizing how important this is (very), I think it might be useful to catalogue some of the other issues that, unfortunately, probably need to be addressed as prerequisites to addressing this one:

At the very least, it seems like these issues would have to be addressed before this one could be (or addressed as part of addressing this one):

1. 1657

2. 2030

Those two issues make it risky to even use composite actions in the same workflow as any action with a post-run step, and presumably would be exacerbated by the composite actions themselves supporting post-run steps.

In addition, the following issues, while not direct dependencies of this one, seem relevant or related:

1. 1947

2. 2009

3. 2418

Note that I'm not suggesting that the issues above are "more important" than this one; just that, someone aiming to address this issue may end up finding that they need to address some or all of the above issues first, before this issue becomes solvable at all.

[Himura2la]

It's so strange that this basic feature is not implemented. I don't want to use JS for writing basic thing easily done in shell. What do you even mean by composite if it's simply a shell action?? The restriction on post-action is super strange, I even see this step in my workflow, but I can't add any code in it

[antoineco]

@Himura2la everybody here wants to see this happen as much as you do. Until this happens, you can use this (as already suggested) so that you don't have to write any Javascript yourself:

    - uses: pyTooling/Actions/with-post-step@v0.4.5
      with:
        main: |
          main shell commands
        post: |
          post execution shell commands

Example of usage here

[umarcor]

It's the same as the action shared by @antoineco but doesn't require a public action - some companies have restrictions on public actions.

For completeness, they can copy the sources to their repo and use it either internally or locally. See https://github.com/pyTooling/Actions/tree/main/with-post-step. It's 10 lines of Apache licensed code.

[philomory]

The with-post-step action is a useful stopgap, but, besides not being as "clean" as a native solution, it's worth pointing out that there are actually situations where it doesn't actually work (and as far as I know, could not be made to work); specifically, if you want to schedule an action (rather than a shell script) to run as post-step of your composite action, with-post-step won't actually help.

The place that this has come up most frequently for me is wanting to have a composite action that executes actions/cache/restore as part of the composite action, and then schedules actions/cache/save as a post-run step (that runs at the end of the calling workflow, not at the end of the composite action). Using actions/cache/restore and actions/cache/save individually gives you a lot more control than just using actions/cache (which is why they exist in the first place), but, using them as part of a composite action is pretty unwieldy, and I don't think it's actually possible to fix that through clever workarounds like with-post-step, it's something that would need native support.

[pauldraper]

I came here to solve the exact same problem as @philomory ... ergonomically and conditionally running cache/restore and cache/save.

[sandstrom]

This would solve a lot of headaches around cache logic and such. Especially since it's not possible to use post steps in Reusable Workflows either.

[zak905]

Hello, any updates about this ?

[junaruga]

We, the Ruby project, are using the gacts/run-and-post-run GitHub Action as alternative at the commit.

[rrauenza]

Another workaround, add this to the top of your run if it is sh/bash:

- name: run container 
  run: |
      function cleanup() {                                                
          docker image rm "${{ steps.build-image.outputs.image }}" || true        
      }                                                                       
      trap cleanup EXIT   
      docker run "${{ steps.build-image.outputs.image }}"

edit: Can anyone explain the thumbs down? Am I mistaken that this works to achieve a post cleanup that always runs?

[jaymecd]

trap works within same shell.

so this is almost equivalent (except for run fails):

      docker run "${{ steps.build-image.outputs.image }}"
      docker image rm "${{ steps.build-image.outputs.image }}" || true        

[RockLobster]

Another workaround, add this to the top of your run if it is sh/bash:

- name: run container 
  run: |
      function cleanup() {                                                
          docker image rm "${{ steps.build-image.outputs.image }}" || true        
      }                                                                       
      trap cleanup EXIT   
      docker run "${{ steps.build-image.outputs.image }}"

edit: Can anyone explain the thumbs down? Am I mistaken that this works to achieve a post cleanup that always runs?

The post step is intented to run once the whole github job is done, not just after the current step/action

[ramizpolic]

Supporting pre and post in composite actions seems like a usable addition. We don't have this on our roadmap at the moment, but I'll add it to the list for future work. Thanks for the suggestion!

Hi @ethomson, any ETA on when this would be available? It's still relevant