actions / runner

The Runner for GitHub Actions :rocket:
https://github.com/features/actions
MIT License
4.91k stars 964 forks source link

"Unrecognized named-value: 'env'. Located at position 1 within expression" when used in reusable workflow jobs #2372

Open nive-copia opened 1 year ago

nive-copia commented 1 year ago

Describe the bug Usage of env in workflow that uses reusable workflow generates "Unrecognized named-value: 'env'. Located at position 1 within expression"

To Reproduce Use the following yml:

name: Test Workflow

on:
  push:

env:
  SOMETHING: 1000

  test:
    name: call workflow
    uses: ./.github/workflows/callee.yml
    secrets: inherit
    with:
      run_url: "{run_url}"
      message: ${{ env.SOMETHING }}

Expected behavior env.SOMETHING is usable and can be passed into reusable workflow

Actual behavior

The workflow is not valid. .github/workflows/create-branch.yml (Line: #, Col: ##): Unrecognized named-value: 'env'. Located at position 1 within expression: env.SOMETHING

Runner Version and Platform

Version of your runner?

OS of the machine running the runner? ubuntu-latest

What's not working?

image

sagi-sensorz commented 6 months ago

just use vars instead of env

This is only relevant for repository level variables. If we use mono repo with multiple workflows and want to define a reusable variable to pass to multiple reusable workflows (e.g. working_directory) within the workflow, there is no solution πŸ€¦β€β™‚οΈ . @whsalazar

AdamJudge commented 6 months ago

+1 Extremely frustrating implementing new ideas benefiting DRY methods of coding. Our workflows have up to 16 jobs, and we only want certain ones to run on certain branches.

We moved from GitLab years ago, and are still missing what we thought would be key/obvious features

lure8225 commented 6 months ago

+1 we intend to have a env variable set in the beginning by evluating some expression and then use this to conditionally trigger jobs. With this problem this will not work and the documentation clearly states that all contexts should be available.

lure8225 commented 6 months ago

Just checked the code for the runner and current impression is that this can not be fixed on runner side as the conditions which jobs get executed is taken server side. A runner simply executes all Jobs it gets, I could not find any conditions checking (found them for steps where this is working fine)

Sadly server part is not open source so hard to support with fixing this

shgnplaatjies commented 5 months ago

I bumped into this; and resolved it by using ${{ needs.your-job-name.outputs.YOUR_NON_SENSITIVE_VAR }}

It's not a long-term solution since it leads to a deprecation warning, nor is it appropriate for sensitive variables:

" The set-output command is deprecated and will be disabled soon. Please upgrade to using Environment Files. For more information see: https://github.blog/changelog/2022-10-11-github-actions-deprecating-save-state-and-set-output-commands/ "

tompreston commented 5 months ago

We were trying to use a repeated container image name. For anyone still searching, this part of the docs (from this comment) explained it for us. The env context is not available in the jobs.<job_id>.container workflow key. Screenshot 2024-06-04 at 09 50 59

It looks like YAML anchors are not supported and there might be a route forwards with repository variables in the vars context, but unfortunately it doesn't follow the same branch protection rules. For example requiring a review to merge to master.

dror-g commented 3 months ago

Guess Github are aware and suggest using output and needs to pass so-called "global" env vars between jobs.. passing matrix def

An extra job is required just to pass the env var into output.
I have strong feelings about this. Negative ones.

Edit: also, even for the matrix example above - this is a huge pain as echo / output completely breaks complex or multiline matrix definitions, that need to be encoded back and forth from string to string to json.. No wonder you gave a simple array as an example.. try multiline. I dare you.

rupert-jung-mw commented 3 months ago

This is highly unexpected and should just work.

dror-g commented 3 months ago

For others that might end up here, this is what worked for me in the interim for multi-line matrix passing between jobs Credit to Manu MagalhΓ£es @magmanu (Thanks!!!)

In Github repo environment I define a multi-line var called ENV_PARAMS_MATRIX:

[
  {
  "name": "service-one",
  "params": {
    "SERVER_URL": "https://dev.example.com",
    "type": "express"
    }
  },
  {
  "name": "service-two",
  "params": {
    "SERVER_URL": "https://prod.example.com",
    "type": "flask"
    }
  }
]
<-- Matrix can also be defined in Workflow itself without repo env vars

``` name: example workflow on: [push, fork] env: AWS_REGION: aq-east-1 ENV_PARAMS_MATRIX: | [ { "name": "service-one", "params": { ... ... ```

Jobs defined as:

jobs:
  build:
    name: Build Image
    runs-on: ubuntu-latest
    steps:

    - name: Checkout build repo
      uses: actions/checkout@v4

    # Rest of job steps .. then this:

    - name: Prepare multiline payload
      id: payload
      run: |
          payload=$(printf '${{ vars.ENV_PARAMS_MATRIX }}')
          echo "PAYLOAD<<EOF"$'\n'"$payload"$'\n'EOF >> "$GITHUB_OUTPUT"
      shell: bash

    outputs:
      payload: ${{ steps.payload.outputs.PAYLOAD }}

  # Then second job uses matrix payload:

  deploy:
    name: Deploy
    needs: [build]
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        myservers: ${{ fromJSON(needs.build.outputs.payload) }}

    steps:

    # Consume matrix params in step
    - name: Fill iserver name in task definition
      uses: aws-actions/amazon-ecs-render-task-definition@v1
      with:
        container-name: ${{ matrix.myservers.name }}-backend

This printf + PAYLOAD<<EOF is the best method I found of formatting multi-line matrix def.

lukian-tabandzhov commented 2 months ago

Hope this could help someone as I recently faced a similar issue and managed to work around it.

@dror-g, @magmanu, @lure8225, a solution with no job that just outputs the matrix value.

The solution is a bit tricky but covers:

Adding a new predefined matrix is a bit tricky as this needs to be kind of a valid JSON string KEY: VALUE but without the opening and closing curly brackets.

name: Predefined Matrix
on:
  workflow_dispatch:
    inputs:
      server_group:
        type: choice
        options:
          - SERVER-GROUP-A
          - SERVER-GROUP-B
        description: 'Select Server Group'

jobs:
  # 1. We define JSON string using the "format" expression as multiline strings as input parameter for fromJSON fails :/
  # 2. The result of the format expression (i.e. JSON string) we provide as an input parameter to the fromJSON expression.
  # 3. We take the value of the user-selected input from the fromJSON result object to pass as a matrix value
  multi-deploy:
    strategy:
      matrix:
        server: ${{
          fromJSON(
            format(
              '{{{0}, {1}}}',
              '"SERVER-GROUP-A":["server A1", "server A2", "server A3"]',
              '"SERVER-GROUP-B":["server B1", "server B2"]',
            )
          )[ inputs.server_group ] }}
    name: Processing ${{ matrix.server }}
    uses: ./REUSABLE-DEPLOYMENT-WORKFLOW.yaml
    with:
      server: ${{ matrix.server }}
esaporski commented 2 months ago

image

anajuliabit commented 2 months ago

please github devs do something 😩

ulidtko commented 2 months ago

@anajuliabit I'm mostly sure that exactly 0 of GH devs are subscribed to every issue and are reading every comment. You'd have to summon their attention via explicit pings.

For example, ping @ericsciple @TingluoHuang

ericsciple commented 2 months ago

The env context is not supported under jobs.<job_id>.with.<with_id>

Refer https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/accessing-contextual-information-about-workflow-runs#context-availability

ericsciple commented 2 months ago

To elaborate a bit more, env is for process environment variables. Also, secrets can be mapped into env, so if secrets isn't allowed, then env isn't allowed either.

Similarly, the secrets context is generally only allowed within a job. It is not allowed within fields that are used by our server to manage the workflow orchestration.

ericsciple commented 2 months ago

Skimming the replies regarding use cases...

One solution might be to check-in a script which sets up your job environment variables. https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/workflow-commands-for-github-actions#setting-an-environment-variable. For scenarios involving run-context or workflow dispatch inputs, $GITHUB_EVENT_PATH might also help.

Not perfect, but might work for many scenarios.

ericsciple commented 2 months ago

Also note, reusable workflows are sort of a security boundary. That is why env values don't flow to reusable workflows. Otherwise callers could influence the jobs in ways unintended by the reusable workflow author. E.g. consider https://github.blog/changelog/2022-03-21-github-actions-restrict-self-hosted-runner-groups-to-specific-workflows/

ulidtko commented 2 months ago

Hi @ericsciple thanks for responding :pray:

I know it's tough to unbundle the assortment of various issues that are being conflated into one. Perhaps you can think of error message improvements that'd send people to corresponding docs explaining that certain usage isn't supported, neither will be? That'd be super helpful to reduce confusion, as apparently there's a lot of difficulty in locating the authoritative docs.

For example, I'm coming from a (prematurely closed) issue #1189 which reports the same error β€” but without reusable workflows, secrets, and under jobs.<job_id>.env (not in jobs.<job_id>.with.<with_id>).

I.e. this does not work, throws the error:

jobs:
  acme-bundle:
    runs-on: ubuntu-20.04
    needs: build
    env:
      # if triggered from a branch containing /, use "wip", otherwise tag name
      version: ${{ contains(github.ref_name, '/') && 'wip' || github.ref_name }}
      fullfilename: acme-${{ env.version }}.${{ github.sha }}.bundle
    steps:
      [...]
      - name: Pack
        run: tar -czf $fullfilename --format posix -C DIST .
      - name: Upload
        run: aws s3 cp $fullfilename s3://$BUCKET/ACME/$version/$fullfilename

#-- NOTE: the workflow author wants both values as env-vars, to be used in shell steps.
#-- NOTE: the second var is defined using the first var ${{ env.version }}

# ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
# The workflow is not valid. .github/workflows/release.yml (Line: 190, Col: 21):
# Unrecognized named-value: 'env'. Located at position 1 within expression: env.version

Can this be helped with somehow?

ericsciple commented 2 months ago

+1 thanks @ulidtko - good feedback. I will forward along internally.

ben5311 commented 4 days ago

This didn't work back in late 2021 when reusable workflows were introduced and it still doesn't work three years later in 2024. See you in 3 years.