Description of vars usage is not acurate

withakay commented 4 months ago

Code of Conduct

[X] I have read and agree to the GitHub Docs project's Code of Conduct

What article on docs.github.com is affected?

https://docs.github.com/en/actions/learn-github-actions/contexts#example-usage-of-the-vars-context

What part(s) of the article would you like to see updated?

To article states "This example workflow shows how configuration variables set at the repository, environment, or organization levels are automatically available using the vars context."

This is not acurate. In the below example where vars.RUNNER is variables defined in the environement dev (that is to say not at the repository or organisation level) then vars.RUNNER will not be evaluated and you can expect an error like:

Error when evaluating 'runs-on' for job 'test-job'. .github/workflows/test-job.yaml (Line: 26, Col: 14): Unexpected value ''

  test-job:
    name: "Can echo Hello World!"
    environment: "dev"
    runs-on: ${{vars.RUNNER}}
    steps:
      - name: hello-world
        run: echo "Hello World!"

It seems to me either the article is incorrect or there is a bug in the github actions engine as the described behaviour is not acurate.

Additional information

To reproduce

create a new workflow

name: "Runner var test"
on:
  workflow_dispatch:

jobs:
  hello-world-job:
    name: "Can echo Hello World!"
    environment: "dev"
    runs-on: ${{vars.RUNNER}}
    steps:
      - name: hello-world
        run: echo "Hello World!"

Create and environment in the repository and add a variable called RUNNER with a value of ubuntu-latest
Run the Workflow and observe it failing in

welcome[bot] commented 4 months ago

Thanks for opening this issue. A GitHub docs team member should be by to give feedback soon. In the meantime, please check out the contributing guidelines.

nguyenalex836 commented 4 months ago

@withakay Thanks so much for opening an issue! I'll get this triaged for review ✨

withakay commented 3 months ago

I edited the description to fix some typos

sunbrye commented 2 months ago

Hi @withakay 👋 Thank you for raising this concern 💛 ``

I've followed your instructions to try to reproduce the issue.

1.) I created a new workflow and ran the workflow without adding the variable RUNNER to the repository. This causes to the workflow to fail as expected.

2.) I added the variable RUNNER with a value of ubuntu-latest to the repository, then ran the same workflow. The workflow succeeded as expected. Attached is the successful log output from the examples provided

2024-07-18T18:23:52.8768312Z Current runner version: '2.317.0'
2024-07-18T18:23:52.8793921Z ##[group]Operating System
2024-07-18T18:23:52.8794615Z Ubuntu
2024-07-18T18:23:52.8795078Z 22.04.4
2024-07-18T18:23:52.8795387Z LTS
2024-07-18T18:23:52.8795718Z ##[endgroup]
2024-07-18T18:23:52.8796163Z ##[group]Runner Image
2024-07-18T18:23:52.8796600Z Image: ubuntu-22.04
2024-07-18T18:23:52.8796996Z Version: 20240714.1.0
2024-07-18T18:23:52.8798179Z Included Software: https://github.com/actions/runner-images/blob/ubuntu22/20240714.1/images/ubuntu/Ubuntu2204-Readme.md
2024-07-18T18:23:52.8799678Z Image Release: https://github.com/actions/runner-images/releases/tag/ubuntu22%2F20240714.1
2024-07-18T18:23:52.8800546Z ##[endgroup]
2024-07-18T18:23:52.8801034Z ##[group]Runner Image Provisioner
2024-07-18T18:23:52.8801482Z 2.0.370.1
2024-07-18T18:23:52.8801819Z ##[endgroup]
2024-07-18T18:23:52.8803271Z ##[group]GITHUB_TOKEN Permissions
2024-07-18T18:23:52.8804960Z Contents: read
2024-07-18T18:23:52.8805393Z Metadata: read
2024-07-18T18:23:52.8806106Z Packages: read
2024-07-18T18:23:52.8806662Z ##[endgroup]
2024-07-18T18:23:52.8809212Z Secret source: Actions
2024-07-18T18:23:52.8809817Z Prepare workflow directory
2024-07-18T18:23:52.9487603Z Prepare all required actions
2024-07-18T18:23:52.9734155Z Complete job name: Can echo Hello World!
2024-07-18T18:23:53.0891633Z ##[group]Run echo "Hello World!"
2024-07-18T18:23:53.0892598Z [36;1mecho "Hello World!"[0m
2024-07-18T18:23:53.1401814Z shell: /usr/bin/bash -e {0}
2024-07-18T18:23:53.1402675Z ##[endgroup]
2024-07-18T18:23:53.1865309Z Hello World!
2024-07-18T18:23:53.2162348Z Cleaning up orphan processes

We will close the issue with no further actions. Thank you for understanding and being a valued part of our community.

withakay commented 2 months ago

@sunbrye

Thanks for taking the time to attempt to reproduce. Unfortunately I think you must have configured something differently. When testing as I am still able to reproduce the issue and as such I don't think this issue should be closed. I note you mentioned creating a variable in the repository but don't mention creating an environment and adding a variable to that?

I have created a repository to demonstrate here:

https://github.com/withakay/runner-env-var-repro

If you could take another look that would be great

nguyenalex836 commented 2 months ago

@withakay Hello! 👋 Given the communication between you and @sunbrye that has already occurred on this issue, I'll reopen this issue and close the duplicate issue you recently submitted 💛

github-actions[bot] commented 2 months ago

Thanks for opening an issue! We've triaged this issue for technical review by a subject matter expert :eyes:

sunbrye commented 2 months ago

@withakay Thank you for taking the time to write up the steps to reproduce the issue. With your steps, I was able to reproduce your issue with the environment-level configuration variable.

The environment-level configuration variable throws an undefined error in the following:

name: "Runner var test"
on:
  workflow_dispatch:

jobs:
  hello-world-job:
    name: "Echo Hello World!"
    environment: "dev"
    runs-on: ${{vars.RUNNER}}
    steps:
      - name: hello-world
        run: echo "Hello World!"

The environment-level configuration variable is passed through in the following:

name: "Runner var test"
on:
  workflow_dispatch:

jobs:
  hello-world-job:
    name: "Echo ubuntu-latest"
    environment: "dev"
    runs-on: "ubuntu-latest"
    steps:
      - name: hello-world
        run: echo ${{vars.RUNNER}}

The request to update the documentation to clarify that the environment-level configuration variable is not available until the target environment is declared makes sense to me. You, or anyone else, are free to open a pull request to make these changes.

beremonsivais commented 1 month ago

There are two different articles which mention the use of configuration variables at the environment level by the vars context without clarifying that they are only available after the environment is declared by the runner:

github / docs