Closed strickvl closed 7 months ago
Hi! You need to activate virtual environments so set VIRTUAL_ENV=./.venv
after creating.
Related #1326
Sorry maybe I'm being dumb here, but I've tried this and it still doesn't pick up the package.
steps:
- name: Checkout code
uses: actions/checkout@v4.1.1
- name: Set up Python
uses: actions/setup-python@v5.0.0
with:
python-version: '3.10'
- name: Install current package as editable
run: |
pip install uv
uv venv
VIRTUAL_ENV=./.venv
uv pip install darglint
- name: Check docstrings
run: bash scripts/docstring.sh
env:
VIRTUAL_ENV: ./.venv
I even threw in a source .venv/bin/activate
after the uv venv
and that also did nothing. Am I doing what you suggested, or am I doing it wrong?
With GitHub Actions I think it makes more sense to install the dependencies globally in the virtual machine. uv does not support that yet. See #1374
In your “Check docstrings” step shown above you’re missing also updating the PATH environment variable to include “$VIRTUAL_ENV/bin”
. Or you can call . .venv/bin/activate
at the beginning of the step, which will take care of the VIRTUAL_ENV and PATH environment variables for you.
Had a similar issue, got this working for me with the following snippet :
angreal-tests-linux:
name: "angreal run-tests linux"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: 1.67.0
override: true
- uses: Swatinem/rust-cache@v1
- uses: webfactory/ssh-agent@v0.7.0
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- uses: actions/setup-python@v4
with:
python-version: "3.12"
- run: echo "VIRTUAL_ENV=${Python_ROOT_DIR}" >> $GITHUB_ENV
- run: pip install uv
- run: uv pip install maturin pytest
- run: uv pip install angreal@.
- run: cargo test -v -- --nocapture --test-threads=1
- run: python -m pytest -svv
Its a bit of a hack, but basically you create the environment variable and set it to the root python and append it to the special github_env variable.
This basically tells uv to just use the system python instead oa virtual env, should be good to go,
@dylanbstorey that's a neat trick, and helpful it works across operating systems too. But would be good for uv
to support installations globally too to make this unnecessary.
Ok and key here is that the - run: echo "VIRTUAL_ENV=${Python_ROOT_DIR}" >> $GITHUB_ENV
step runs as a separate step. i.e. you can't just run that bash command from within a step that has some other uv
commands as then it won't pick up the Env var.
Was running into race conditions here or at least issues with steps / jobs running in parallel attempting to access the same environment, it seems. I switched back to using source ...
everywhere as it seemed more controlled.
Ideally for me there would be clear instructions on good practice for using uv
in GitHub Actions, including on Windows, and including with a cache using actions/cache
.
Thank you @dylanbstorey .
In my use of your trick, I quote "$GITHUB_ENV"
as per examples in https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-an-environment-variable and as advised by shellcheck.
SC2086:info:1:42: Double quote to prevent globbing and word splitting [shellcheck]
Ideally for me any documented example will pass actionlint
.
Can confirm @dylanbstorey's hack
- run: echo "VIRTUAL_ENV=${Python_ROOT_DIR}" >> $GITHUB_ENV
is working fine in my case:
The previous solutions suggested in this thread aren't cross-platform:
echo
is a Unix-only command; you need different syntax to set an environment variable on WindowsVIRTUAL_ENV
to the Python root dir without creating a virtual environment doesn't work on Windows, due to the fact that python.exe
isn't in a Scripts/
directory on Windows if you don't have a virtual environment activated, and uv
isn't (yet?) aware of that.All that means that the previous suggested solutions won't work in a workflow job that runs on multiple operating systems in CI. After much debugging with @MichaReiser, here's a workflow job that I'm now successfully using to run tests for my hobby project typeshed-stats
. It runs on Linux, MacOS and Windows in CI:
jobs:
pytest-tests:
name: Run tests with pytest
runs-on: ${{ matrix.os }}
strategy:
matrix:
os: ["ubuntu-latest", "windows-latest", "macos-latest"]
python-version: ["3.10", "3.11", "3.12"]
fail-fast: false
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }} on ${{ matrix.os }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
allow-prereleases: true
- name: Create and activate a virtual environment (Windows)
if: ${{ runner.os == 'Windows' }}
# uv doesn't (yet) allow us to install packages globally;
# we have to create and activate a virtual environment
run: |
irm https://astral.sh/uv/install.ps1 | iex
uv venv .venv
"VIRTUAL_ENV=.venv" | Out-File -FilePath $env:GITHUB_ENV -Append
"$PWD/.venv/Scripts" | Out-File -FilePath $env:GITHUB_PATH -Append
- name: Create and activate a virtual environment (Unix)
if: ${{ runner.os != 'Windows' }}
# uv doesn't (yet) allow us to install packages globally;
# we have to create and activate a virtual environment
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv .venv
echo "VIRTUAL_ENV=.venv" >> $GITHUB_ENV
echo "$PWD/.venv/bin" >> $GITHUB_PATH
- name: Install dependencies
run: uv pip install -e ".[pytest]"
- run: uv pip freeze
- name: Run tests under coverage
run: |
coverage run -m pytest
coverage report
@AlexWaygood, @MichaReiser Thank you for investigating this.
If you want to use echo
on Windows in GitHub Actions, you can set shell: bash
on a GitHub Action.
If you want to use
echo
on Windows in GitHub Actions, you can setshell: bash
on a GitHub Action.
@adamtheturtle we tried that but it caused other things to break ;) take a look at the test failures on https://github.com/AlexWaygood/typeshed-stats/pull/191.
Had / having similar issues on our CI. Choosing to wait for a more stable way for this global install to happen which I believe will be worked on next week.
The following worked here.
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
steps:
- uses: actions/checkout@v3
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install uv
uv venv
uv pip install -e .[dev]
- name: Activate virtualenv
run: |
. .venv/bin/activate
echo PATH=$PATH >> $GITHUB_ENV
- name: Test with pytest
run: pytest
Ideally for me there would be clear instructions on good practice for using uv in GitHub Actions, including on Windows, and including with a cache using actions/cache.
I'll just note that setup-python
action also has a builtin cache. @alexwaygood have you done any testing to see if using this cache meaningfully speeds things up? Given how fast uv
is, I wouldn't be surprised if there wasn't much difference. :-)
@AlexWaygood have you done any testing to see if using this cache meaningfully speeds things up?
Nope! But it's a good point!
Given how fast
uv
is, I wouldn't be surprised if there wasn't much difference. :-)
I think that would likely depend quite a lot on how complex it is to resolve the requirements set. But for simple requirements sets, I'd wager you're likely right.
on how complex it is to resolve the requirements set.
The way I understand it, the cache does not really speeds up resolution itself, no? Well, unless you do not run uv pip install
at all upon cache hit.
The way I understand it, the cache does not really speeds up resolution itself, no? Well, unless you do not run
uv pip install
at all upon cache hit.
oh I don't actually really know anything about how the caching works. Should probably not have implied that I did 😆
Closing this issue as the --system
flag has basically solved this issue for me. Thanks uv
team!
The cache does improve resolution times because sometimes we need to build source distributions to determine their requirements during resolution which is generally slow — the cache contains these builds.
Hi! For those of you who are subscribed to this and are using echo "VIRTUAL_ENV=${Python_ROOT_DIR}"
as a work-around — as of 0.2.0 this is no longer supported and we'd recommend using the --system
flag or --python <path>
instead.
Can anyone share an example of how to set up uv in GitHub actions with --system/--python?
@elronbandel here's an example:
Having trouble converting our pre-existing CI workflows / scripts to use
uv
, as you can see here. Problem relates to the virtual environments which don't seem to be retained between workflow steps.This is maybe the easiest illustration of that:
One step installs it, and then the next step runs a script which in turn attempts to use the package, but it fails because the CI can't be found.
I don't see any docs yet, so wondering how best to handle this?