actions / runner

The Runner for GitHub Actions :rocket:
https://github.com/features/actions
MIT License
4.77k stars 932 forks source link

Allow to unset env variables #1126

Open mpdude opened 3 years ago

mpdude commented 3 years ago

echo 'SOME_VAR=some-value' >> $GITHUB_ENV can be used to set an environment variable for subsequent steps, but there seems to be no documented way to unset it.

Note that echo 'SOME_VAR=' >> $GITHUB_ENV will set it to the empty value, but that's different from not having the variable set at all.

Rationale: In some workflows, it might be necessary to checkout and/or run untrusted code. At that point I'd like to clean up as much sensitive values as possible. Unsetting env vars (like SSH_AGENT_PID or SSH_AUTH_SOCK) is part of this.

fhammerl commented 3 years ago

Thank you for reporting this issue. Would you mind providing an example workflow to illustrate the issue?

mpdude commented 3 years ago

Sure! It's not a full workflow, but I hope it gets the message across...?

            -   uses: aws-actions/configure-aws-credentials@v1
                with:
                    aws-access-key-id: ${{ secrets.ACCESS_KEY_ID }}
                    aws-secret-access-key: ${{ secrets.SECRET_ACCESS_KEY }}
            -   id: login-ecr
                uses: aws-actions/amazon-ecr-login@v1
            -   run: docker pull some-private/image-from-ECR
            -   run: yarn install

I don't trust yarn install specifically, especially with regard to lifecycle scripts. --ignore-scripts doesn't entirely cut it because sooner or later I will have to run at least some of them (details).

So, basically what I would like to do is to clean up as much as possibly after I've done the parts that I trust more, before I start running the "risky" things.

There is not always an option of splitting such tasks across separate workflows, which would be the best solution obviously.

mpdude commented 3 years ago

~Wait… could I just remove lines from $GITHUB_ENV (sed …) and that would do it…?~

No. The file seems to be empty at the beginning of each workflow step.

ethomson commented 2 years ago

This seems like a useful addition. I'll put this on the backlog. Thanks for the feedback.

chrisd8088 commented 1 year ago

Another use case example for this functionality turned up while working on git-lfs/git-lfs#5236, namely, that the ruby/setup-ruby@v1 action sets the TMPDIR environment variable for subsequent steps, but for the Git LFS project's CI test suite, we need to unset it before running various non-Ruby test scripts. Otherwise, we find that mktemp(1) creates temporary directories under the TMPDIR location but Go language programs using os.TempDir() create them elsewhere, on another volume, and the files cannot then be moved between the two locations.

Resolving this issue while still upgrading to the current ruby/setup-ruby@v1 action from the deprecated actions/setup-ruby@v1 action required us to use env -u TMPDIR in multiple locations. It would be much easier and more robust if we could simply unset a given environment variable by writing once to a GitHub Actions environment file, like the one specified by GITHUB_ENV, as soon as we're done with our Ruby-related actions and want to move on to our Go-related actions.

Constantin07 commented 1 year ago

I'm facing the same issue - I need to cleanup AWS creds from the aws-actions/configure-aws-credentials step but don't know how to do it properly.

michielvermeir commented 1 year ago

I'm facing the same issue - I need to cleanup AWS creds from the aws-actions/configure-aws-credentials step but don't know how to do it properly.

Yep, that'd also be my use case. The problem for me is that environment variables exported by aws-actions/configure-aws-credentials always have a higher precedence in the AWS CLI credentials chain than if I were to set AWS_PROFILE.

AlexAtkinson commented 1 year ago

For GITHUB_ENV, you can tear out content with sed. Note specifically that this is simply a file that's sourced, so you can work on it in the same way you might a .bashrc file.

sed -i "/badvar=/d" $GITHUB_ENV

I cut a gist to help explain these convenience files. https://gist.github.com/AlexAtkinson/f9086c4a88a28cfdce5dbb1b15e947ea

mpdude commented 1 year ago

At least two years ago, that did not work:

https://github.com/actions/runner/issues/1126#issuecomment-853046696

kmcentush commented 1 year ago

Has anyone found a workaround? I also was not able to make sed work.

Brikaa commented 1 year ago

I believe the sed solution won't work with multiline values

Moulick commented 1 year ago

This is becoming a big issue when we want to utilize some third-party action that sets some env variables that then we cannot unset. This becomes an even bigger issue when we want to utilize that action several times and handle the output in a custom way.

I am writing a composite action to invoke aws-actions/configure-aws-credentials@v3 multiple times and stuff the creds into separate profiles. But I cannot unset the AWS_* variables being set by the action 😢. This leads to following steps still having at-least AWS_SESSION_TOKEN="" and create problems

EDIT: https://github.com/marketplace/actions/configure-multiple-aws-roles here is the action

mossad-zika commented 9 months ago

please help

francardoso93 commented 9 months ago

Using sed still doesn't work, as explained in this comment: https://github.com/actions/runner/issues/1126#issuecomment-853046696

bryanmacfarlane commented 9 months ago

Yeah, you can't unset because each step is run out of proc and the toolkits export variable function informs the runner on variables to set on future steps: https://github.com/actions/toolkit/tree/main/packages/core#exporting-variables

I believe it will also short circuit if the value is "" but might be worth validating. Would probably need an unset toolkit / runner support. This all does create odd coupling between steps though :/

NRodrigues001 commented 7 months ago

Can someone post a link to the list where we can add customers who are asking for this feature? This is a feature Experian is very interested in. I didn't see one in the issue/discussion. Thanks!

cc @yasiralibrahem

kellertk commented 7 months ago

Does something like this work?

- uses: actions/github-script@v7
  id: unset-env-vars
  with:
  script: |
    const variables = [
      'ENV_VAR_NAME',
    ];

    for (const variable of variables) {
      if (process.env[variable]) {
        core.debug(`${variable} is set, removing`);
        delete process.env[variable];
      }
    }
Moulick commented 7 months ago

@kellertk nope, does not work :(

chrisgustavsen-wf commented 4 months ago

@bryanmacfarlane We are encountering this issue as well. For example - https://github.com/awslabs/aws-sdk-kotlin/issues/1281. Any updates on priority or workarounds?

michaelschuett-tomtom commented 2 months ago

not a real work around but I was able to do this which at least works for my usecase.

    - name: Set environment
      shell: sh
      run: |
          set -e
          cp $GITHUB_ENV /tmp/backup
          echo "SECRET=${{ inputs.secret }}" >> $GITHUB_ENV

then after you are done you can simply mv /tmp/backup $GITHUB_ENV

runlevel5 commented 2 months ago

@kellertk yours does not work simply because you only unset the environment variables and NOT modify the GITHUB_ENV file so the next step would reload the environment variables from that file again.

runlevel5 commented 2 months ago

The sed -i "/badvar=/d" $GITHUB_ENV suggestion from @AlexAtkinson won't work too because GitHub has disallowed any operation to do with that file. Yes, even if you cp $GITHUB_ENV /tmp/my_temp, GitHub would not allow you to work with that copy.

I guess it is all about security, still it is ANNOYING that GitHub does not offer a way

jebeaudet commented 1 month ago

Adding my voice to the problem, we have an action to fetch a secured credential using OIDC like this(yaml simplified for demo purposes) :

    - name: Assume OIDC role
      uses: aws-actions/configure-aws-credentials@v4
      with:
        role-to-assume: arn:aws:iam::1234:role/some-secured-role
        aws-region: us-east-2

    - name: Get Secret value
      id: get_secret_value
      shell: bash
      env:
        AWS_REGION: us-east-2
      run: |
          export SECRET_VALUE=$(\
            aws secretsmanager get-secret-value \
            --secret-id arn:aws:secretsmanager:us-east-2:1234:secret:/some/secret \
            --query SecretString \
            --output text \
          )
          if [[ "${{ inputs.jq_escape }}" == "true" ]]; then
            SECRET_VALUE=$(echo "$SECRET_VALUE" | jq -c .)
          fi
          echo "::add-mask::$SECRET_VALUE"
          echo "secret-value=$SECRET_VALUE" >> "$GITHUB_OUTPUT"

    - name: Unassume OIDC role and environment variables cleanup
      shell: bash
      run: |
        echo "AWS_ACCESS_KEY_ID=" >> $GITHUB_ENV
        echo "AWS_SECRET_ACCESS_KEY=" >> $GITHUB_ENV
        echo "AWS_SESSION_TOKEN=" >> $GITHUB_ENV
        echo "AWS_DEFAULT_REGION=" >> $GITHUB_ENV
        echo "AWS_REGION=" >> $GITHUB_ENV

This works fine to clear out the creds, but we're left with AWS_REGION and AWS_DEFAULT_REGION that are set with empty value. Boto3 and the Java AWS sdk do not like this and we end up with errors like

Caused by: java.lang.IllegalArgumentException: Request endpoint must have a valid hostname, but it did not: https://ssm/..amazonaws.com
File "botocore/endpoint.py", line 400, in create_endpoint
ValueError: Invalid endpoint: https://sts..amazonaws.com/

I know I could run this step in another job, but for simplicity and speed purposes, it would be much simpler to be able to properly unset environment variables. My only workaround for this is to set a default region at the end of the action but this does not make sense at all.

jebeaudet commented 1 month ago

I've dug a bit in the codebase and it seems that on each step, the runner service will generate a new empty set_env_${someGuid} in the /opt/actions-runner/_work/_temp/_runner_file_commands. The step can add variable in this file and they will get processed at the end of the step by the runner service here I think. This is feeded into a global map that is then passed along on each further step.

This is why the sed or cp option does not work, the GITHUB_ENV file only holds new variables that will get propagated downstream. It is always empty at the beginning of each step.

Now the only way I see to do this would be to have a step before aws-actions/configure-aws-credentials that would use something like inotify or other options to watch the files in this folder and overwrite them, before the configure-aws-credentials step completes, but oh man that would be a solid hack.