prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
2.88k stars 156 forks source link

`pixi install` / `pixi run` doesn't handle multiple concurrent invocations very well #1482

Open pavelzw opened 3 months ago

pavelzw commented 3 months ago

In our pre-commit configuration, we use something like

exclude: (^env/|.pixi/)
repos:
  - repo: local
    hooks:
      # prettier
      - id: prettier
        name: prettier
        entry: pixi run -e lint prettier --write --list-different --ignore-unknown
        language: system
        types: [text]
        files: \.(md|yml|yaml)$
      # pre-commit-hooks
      - id: trailing-whitespace-fixer
        name: trailing-whitespace-fixer
        entry: pixi run -e lint trailing-whitespace-fixer
        language: system
        types: [text]
      - id: end-of-file-fixer
        name: end-of-file-fixer
        entry: pixi run -e lint end-of-file-fixer
        language: system
        types: [text]
      - id: check-merge-conflict
        name: check-merge-conflict
        entry: pixi run -e lint check-merge-conflict --assume-in-merge
        language: system
        types: [text]
      # typos
      - id: typos
        name: typos
        entry: pixi run -e lint typos --force-exclude
        language: system
        types: [text]
        require_serial: true
      # cargo fmt and clippy
      - id: cargo-fmt
        name: cargo-fmt
        entry: pixi run -e default cargo fmt --
        language: system
        require_serial: false
        types: [rust]
      - id: cargo-clippy
        name: cargo-clippy
        entry: pixi run -e default cargo clippy --all-targets --all-features --workspace -- -D warnings
        pass_filenames: false
        language: system
        require_serial: false
        types: [rust]
      # taplo
      - id: taplo
        name: taplo
        entry: pixi run -e lint taplo format
        language: system
        types: [toml]

since it is a bad idea to let pre-commit manage lint dependencies when you have a proper package manager who can do that as well in a lint environment.

When running pre-commit run -a, it invokes a lot of pixi run ... processes that all try to install the environment if it doesn't exist. This then leads to warnings/errors like this:

image

To be fair, how our pre-commit config looks is a bit hacky but I think pixi could write a lock file to .pixi to prevent parallel access.


Also, sort-of related, i'm not sure why I see a updating environment 'default' here since the default environment should already be installed (from the setup-pixi step):

image

Any ideas why this is the case?

pavelzw commented 2 months ago

a workaround for this is to add the following at the beginning of your .pre-commit-config.yaml:

      # ensure pixi environments are up to date
      # workaround for https://github.com/prefix-dev/pixi/issues/1482
      - id: pixi-install
        name: pixi-install
        entry: pixi install -e default -e lint
        language: system
        always_run: true
        require_serial: true
        pass_filenames: false