actions / setup-python

Set up your GitHub Actions workflow with a specific version of Python
MIT License
1.74k stars 549 forks source link

All packages with CLI failing with '/home/runner/.local/bin' which is not on PATH #99

Closed heitorlessa closed 2 years ago

heitorlessa commented 4 years ago

Describe the bug

Packages entrypoints/CLI are not available in PATH e.g. flake8, poetry, pytest.

When investigating Github Action logs I can see two noticeable warnings that explains the issue:

As I have another repo using this same action, and it is working just fine, I wonder if something happened recently to cause this - Virtualenv manipulation, PATH manipulation, etc.

Which version of the action are you using?

Tried both - same issue

Environment

If applicable, please specify if you're using a container

Python Versions Please list all of the effected versions of Python (3.8.2, etc.)

3.6, 3.7, 3.8

To Reproduce

Create a new Github action using the following steps:

    steps:
      - uses: actions/checkout@v1
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v1
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: |
          pip install --upgrade pip poetry
          poetry install

It will fail when it attempts to run poetry command. Same happens with any other package.

Run/Repo Url If applicable, and if your repo/run is public, please include a URL so it is easier for us to investigate.

https://github.com/aws-samples/cookiecutter-aws-sam-pipeline/pull/10/checks?check_run_id=730151630

Screenshots If applicable, add screenshots to help explain your problem.

image

Additional context

PATH content - Captured via tmate session

echo $PATH
/opt/hostedtoolcache/Python/3.6.10/x64/bin:/opt/hostedtoolcache/Python/3.6.10/x64:/usr/share/rust/.cargo/bin:/home/runner/.config/composer/vendor/bin:/home/runner/.dotnet/tools:/snap/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/home/linuxbrew/.linuxbrew/bin:/home/linuxbrew/.linuxbrew/sbin:/home/runner/.dotnet/tools

Debug logs

Defaulting to user installation because normal site-packages is not writeable
Requirement already up-to-date: pip in /opt/hostedtoolcache/Python/3.6.10/x64/lib/python3.6/site-packages (20.1.1)
runner@fv-az78:~/work/cookiecutter-aws-sam-pipeline/cookiecutter-aws-sam-pipeline$ pip install flake8
Defaulting to user installation because normal site-packages is not writeable
Collecting flake8
  Downloading flake8-3.8.2-py2.py3-none-any.whl (72 kB)
     |████████████████████████████████| 72 kB 1.7 MB/s
Collecting mccabe<0.7.0,>=0.6.0
  Downloading mccabe-0.6.1-py2.py3-none-any.whl (8.6 kB)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /home/runner/.local/lib/python3.6/site-packages (from flake8) (1.1.3)
Collecting pycodestyle<2.7.0,>=2.6.0a1
  Downloading pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB)
     |████████████████████████████████| 41 kB 796 kB/s
Collecting pyflakes<2.3.0,>=2.2.0
  Downloading pyflakes-2.2.0-py2.py3-none-any.whl (66 kB)
     |████████████████████████████████| 66 kB 9.7 MB/s
Requirement already satisfied: zipp>=0.5 in /home/runner/.local/lib/python3.6/site-packages (from importlib-metadata; python_version < "3.8"->flake8) (3.1.0)
Installing collected packages: mccabe, pycodestyle, pyflakes, flake8
  WARNING: The script pycodestyle is installed in '/home/runner/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script pyflakes is installed in '/home/runner/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The script flake8 is installed in '/home/runner/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.

Add any other context about the problem here.

reillysiemens commented 4 years ago

For what it's worth, I think I ran into a similar issue. Looks like for some reason Python 2 is being used in the container even when I specified another version. Maybe that's related? :thinking:

reillysiemens commented 4 years ago

Ah. Please disregard my earlier comment. Turns out I'd made a typo in my action. I pluralized my matrix with

python-versions: [3.6, 3.7, 3.8, 3.9]

instead of

python-version: [3.6, 3.7, 3.8, 3.9]

I imagine that causes ${{ matrix.python-version }} to be empty then, so Python 2.7 was the default python. :man_shrugging:

indirectlylit commented 4 years ago

I'm reproducing a very similar issue. I have the same Python action running in two repos, and it succeeds in one while failing in another.

In this case I'm trying to run flake8, but get the same "command not found" issue with black and pytest:

name: Python tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: [3.6]
    steps:
    - uses: actions/checkout@v2
    - run: |
        # Hack to get setup-python to work on act
        # https://github.com/nektos/act/issues/251#issuecomment-706412948
        if [ ! -f "/etc/lsb-release" ] ; then
          echo "DISTRIB_RELEASE=18.04" > /etc/lsb-release
        fi
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install flake8
    - name: Lint with flake8
      run: |
        flake8 .

In the repo where this doesn't work, it gives the warning:

WARNING: The script flake8 is installed in '/opt/hostedtoolcache/Python/3.6.12/x64/lib/my_project' which is not on PATH.

In the repo where it does work, there is no warning.

Interestingly, I can reproduce this both in the github runner, and also locally using the act local Docker runner on mac. This implies that there's something about the structure of the two repos which is causing different behaviors. Will try to narrow it down to a reproducible scenario.

indirectlylit commented 4 years ago

Update:

I've narrowed down the issue to a setup.cfg file in the root directory of the affected repo. There was a config like:

[install]
install-scripts=...

which was affecting the location to which pip installed the commands

@heitorlessa looks like your repo might have a similar issue:

https://github.com/aws-samples/cookiecutter-aws-sam-pipeline/blob/f50dfee4f1dd2e618bd3c23098ce0e8b679d5c77/setup.cfg#L1-L2

[install]
prefix= 

A workaround is to cd out of the repo directory before installing dependencies:

    - name: Install dependencies
      run: |
        cd /
        python -m pip install --upgrade pip
        pip install flake8
abitrolly commented 3 years ago

Same here. /home/runner/.local/bin is not added to $PATH. This dir is not present on Fedora 33 either, but this doesn't really make sense for a build system.

  Creating /home/runner/.local/bin
  changing mode of /home/runner/.local/bin/coverage to 755
  changing mode of /home/runner/.local/bin/coverage-3.8 to 755
  changing mode of /home/runner/.local/bin/coverage3 to 755
januszm commented 3 years ago

It may happen in the case where python dependencies are mistakingly referenced BEFORE Set Up Python step, i.e. /home/runner/.local/bin not in PATH error will happen with the following:

    - name: Install dependencies
      run: |
        sudo apt-get -y install ...stuff...
        python -m pip install --upgrade pipenv
        pipenv sync --dev
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}       

But when you install Python dependencies after setting-up Python it'll be OK:

    - name: Install dependencies
      run: sudo apt-get -y ...stuff...
    - name: Set up Python
      uses: actions/setup-python@v2
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install Python dependencies
      run: |
        python -m pip install --upgrade pipenv
        pipenv sync --dev

I just tested with ubuntu-latest and matrix: python-version: ['3.9'] and it works fine.

nikita-bykov commented 2 years ago

Hello @heitorlessa! We apologize for such a late reply. Is this issue still relevant to you? I tried to reproduce it and everything works for me as expected.

nikita-bykov commented 2 years ago

Hello @heitorlessa, I'm going to close the issue because we haven't received a response from you. If the issue is still relevant, feel free to contact us. Thank you!

heitorlessa commented 2 years ago

Yes please do. I moved away from trying again as I couldn't reproduce it either so killed the project

SC-One commented 2 weeks ago

I don't think this problem has been solved! Please reopen it again! Im using requirements:

wheel
setuptools
conan==1.52.0

but in next run shell(bash) Im facing the error: conan not found!

even adding this line before using conan , does not help! echo "/home/runner/.local/bin" >> $GITHUB_PATH

EDIT: I think the problem is that using pip instead pip3! (how I change by default installation packages? do we have any input?)