prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
3.18k stars 174 forks source link

Mismatched virtual package error when using official Docker image #1293

Closed mjkanji closed 5 months ago

mjkanji commented 5 months ago

Checks

Reproducible example

I'm trying to use the official Pixi Docker image for my GitLab CI pipeline, but I'm running into the same error as #346.

Since I'm using the official Docker image, I'm not sure why the version of Linux is not up-to-date.

Here are my config files:

.gitlab-ci.yml ```yml stages: - test precommit-checks: stage: test image: ghcr.io/prefix-dev/pixi variables: PRE_COMMIT_HOME: ${CI_PROJECT_DIR}/.cache/pre-commit GIT_FETCH_EXTRA_FLAGS: $CI_DEFAULT_BRANCH --prune cache: paths: - ${PRE_COMMIT_HOME} before_script: # Install git because the Pixi container doesn't have it - apt update && apt install -y --no-install-recommends git script: - pixi --version - pixi install - pixi run pre-commit --version # Run pre-commit hooks only on the files that are different compared to the default (master) branch. - pixi run pre-commit run --from-ref origin/$CI_DEFAULT_BRANCH --to-ref $CI_COMMIT_SHORT_SHA allow_failure: true # Allow the job to fail without blocking downstream stages ```
pixi.toml ```toml [project] name = "my_project" version = "0.1.0" description = "Add a short description here" authors = ["Muhammad Jarir Kanji "] channels = ["conda-forge"] platforms = ["linux-64", "osx-arm64"] [tasks] dbt_run = { cmd = "dbt run" } [dependencies] python = ">=3.10,<3.11" pip = "*" pre-commit = "*" pytz = "*" pandas = ">=2.2.2,<2.3" # Needed by dbt-coves pyarrow = "*" # Needed by dbt-coves sqlalchemy = "*" [pypi-dependencies] dbt-core = { version = ">=1.7, <1.8" } dbt-redshift = "*" sqlfluff = "*" sqlfluff-templater-dbt = "*" dbt-coves = "*" [feature.dev.dependencies] [environments] dev = { features = ["dev"], solve-group = "dev" } prod = { features = [], solve-group = "dev" } ```

And here's the error I'm getting in the CI job:

$ pixi --version
pixi 0.20.1
$ pixi install
Project in /xx/xx/xx/xx is ready to use!
$ pixi run pre-commit --version
  × The current system has a mismatching virtual package. The project requires
  │ '__linux' to be at least version '5.10' but the system has version
  │ '5.4.109'

Issue description

As described above.

Expected behavior

That the official Docker container has the correct system requirements.

wolfv commented 5 months ago

I think this mainly depends on the "base-image" linux version. Could you try the ubuntu noble based one (24.04)?

That would be pixi:0.20.1-noble or pixi:noble

mjkanji commented 5 months ago

@wolfv That didn't really make a difference. Please see the logs from the CI job below:

CI Logs ```shell Running with gitlab-runner 16.11.0~pre.21.gaa21be2d (aa21be2d) on blue-2.saas-linux-small-amd64.runners-manager.gitlab.com/default XxUrkriX, system ID: s_f46a988edce4 feature flags: FF_USE_IMPROVED_URL_MASKING:true Resolving secrets 00:00 Preparing the "docker+machine" executor 00:06 Using Docker executor with image ghcr.io/prefix-dev/pixi:noble ... Pulling docker image ghcr.io/prefix-dev/pixi:noble ... Using docker image sha256:dbe28428c30f3aa7995a024e1b8f176f302a19e5f9c1aaf32c9574e6a3fb4de7 for ghcr.io/prefix-dev/pixi:noble with digest ghcr.io/prefix-dev/pixi@sha256:8310541161ad8bc72614a0614a7f74d430bba6f4156790ee07cffe6b6afe5b03 ... Preparing environment 00:01 Running on runner-xxurkrix-project-56304299-concurrent-0 via runner-xxurkrix-s-l-s-amd64-1714371199-33ec7fcb... Getting source from Git repository 00:02 Fetching changes with git depth set to 20... Initialized empty Git repository in /builds/xx/xx/xx/ Created fresh repository. From https://xx/xx/xx/ * [new ref] xx -> xx * [new branch] xx -> origin/xx * branch xx -> FETCH_HEAD * [new branch] xx -> origin/master * [new tag] xx -> xx * [new tag] xx -> xx Checking out 3b6aac9d as detached HEAD (ref is xxx)... Skipping Git submodules setup $ git remote set-url origin "${CI_REPOSITORY_URL}" Restoring cache 00:08 Checking cache for default-non_protected... Downloading cache from https://storage.googleapis.com/gitlab-com-runners-cache/xxx/xxxx Successfully extracted cache Executing "step_script" stage of the job script 00:30 Using docker image sha256:dbe28428c30f3aa7995a024e1b8f176f302a19e5f9c1aaf32c9574e6a3fb4de7 for ghcr.io/prefix-dev/pixi:noble with digest ghcr.io/prefix-dev/pixi@sha256:8310541161ad8bc72614a0614a7f74d430bba6f4156790ee07cffe6b6afe5b03 ... $ apt update && apt install -y --no-install-recommends git WARNING: apt does not have a stable CLI interface. Use with caution in scripts. Get:1 http://archive.ubuntu.com/ubuntu noble InRelease [256 kB] Get:2 http://archive.ubuntu.com/ubuntu noble-updates InRelease [89.7 kB] Get:3 http://security.ubuntu.com/ubuntu noble-security InRelease [90.7 kB] Get:4 http://archive.ubuntu.com/ubuntu noble-backports InRelease [90.8 kB] Get:5 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [117 kB] Get:6 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1808 kB] Get:7 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [19.3 MB] Get:8 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [331 kB] Fetched 22.1 MB in 1s (19.8 MB/s) Reading package lists... Building dependency tree... Reading state information... All packages are up to date. WARNING: apt does not have a stable CLI interface. Use with caution in scripts. Reading package lists... Building dependency tree... Reading state information... The following additional packages will be installed: git-man libbrotli1 libcurl3t64-gnutls libdb5.3t64 liberror-perl libexpat1 libgdbm-compat4t64 libgdbm6t64 libgssapi-krb5-2 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap2 libnghttp2-14 libperl5.38t64 libpsl5t64 librtmp1 libsasl2-2 libsasl2-modules-db libssh-4 perl perl-modules-5.38 Suggested packages: gettext-base git-daemon-run | git-daemon-sysvinit git-doc git-email git-gui gitk gitweb git-cvs git-mediawiki git-svn gdbm-l10n krb5-doc krb5-user perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl make libtap-harness-archive-perl Recommended packages: ca-certificates patch less ssh-client krb5-locales libldap-common publicsuffix libsasl2-modules netbase The following NEW packages will be installed: git git-man libbrotli1 libcurl3t64-gnutls libdb5.3t64 liberror-perl libexpat1 libgdbm-compat4t64 libgdbm6t64 libgssapi-krb5-2 libk5crypto3 libkeyutils1 libkrb5-3 libkrb5support0 libldap2 libnghttp2-14 libperl5.38t64 libpsl5t64 librtmp1 libsasl2-2 libsasl2-modules-db libssh-4 perl perl-modules-5.38 0 upgraded, 24 newly installed, 0 to remove and 0 not upgraded. Need to get 15.8 MB of archives. After this operation, 82.6 MB of additional disk space will be used. Get:1 http://archive.ubuntu.com/ubuntu noble/main amd64 perl-modules-5.38 all 5.38.2-3.2build2 [3110 kB] Get:2 http://archive.ubuntu.com/ubuntu noble/main amd64 libdb5.3t64 amd64 5.3.28+dfsg2-7 [734 kB] Get:3 http://archive.ubuntu.com/ubuntu noble/main amd64 libgdbm6t64 amd64 1.23-5.1build1 [34.4 kB] Get:4 http://archive.ubuntu.com/ubuntu noble/main amd64 libgdbm-compat4t64 amd64 1.23-5.1build1 [6710 B] Get:5 http://archive.ubuntu.com/ubuntu noble/main amd64 libperl5.38t64 amd64 5.38.2-3.2build2 [4873 kB] Get:6 http://archive.ubuntu.com/ubuntu noble/main amd64 perl amd64 5.38.2-3.2build2 [231 kB] Get:7 http://archive.ubuntu.com/ubuntu noble/main amd64 libexpat1 amd64 2.6.1-2build1 [87.0 kB] Get:8 http://archive.ubuntu.com/ubuntu noble/main amd64 libkrb5support0 amd64 1.20.1-6ubuntu2 [33.5 kB] Get:9 http://archive.ubuntu.com/ubuntu noble/main amd64 libk5crypto3 amd64 1.20.1-6ubuntu2 [81.7 kB] Get:10 http://archive.ubuntu.com/ubuntu noble/main amd64 libkeyutils1 amd64 1.6.3-3build1 [9490 B] Get:11 http://archive.ubuntu.com/ubuntu noble/main amd64 libkrb5-3 amd64 1.20.1-6ubuntu2 [347 kB] Get:12 http://archive.ubuntu.com/ubuntu noble/main amd64 libgssapi-krb5-2 amd64 1.20.1-6ubuntu2 [142 kB] Get:13 http://archive.ubuntu.com/ubuntu noble/main amd64 libnghttp2-14 amd64 1.59.0-1build4 [73.9 kB] Get:14 http://archive.ubuntu.com/ubuntu noble/main amd64 libpsl5t64 amd64 0.21.2-1.1build1 [57.1 kB] Get:15 http://archive.ubuntu.com/ubuntu noble/main amd64 libbrotli1 amd64 1.1.0-2build2 [331 kB] Get:16 http://archive.ubuntu.com/ubuntu noble/main amd64 libsasl2-modules-db amd64 2.1.28+dfsg1-5ubuntu3 [20.3 kB] Get:17 http://archive.ubuntu.com/ubuntu noble/main amd64 libsasl2-2 amd64 2.1.28+dfsg1-5ubuntu3 [53.2 kB] Get:18 http://archive.ubuntu.com/ubuntu noble/main amd64 libldap2 amd64 2.6.7+dfsg-1~exp1ubuntu8 [195 kB] Get:19 http://archive.ubuntu.com/ubuntu noble/main amd64 librtmp1 amd64 2.4+20151223.gitfa8646d.1-2build7 [56.3 kB] Get:20 http://archive.ubuntu.com/ubuntu noble/main amd64 libssh-4 amd64 0.10.6-2build2 [188 kB] Get:21 http://archive.ubuntu.com/ubuntu noble/main amd64 libcurl3t64-gnutls amd64 8.5.0-2ubuntu10 [333 kB] Get:22 http://archive.ubuntu.com/ubuntu noble/main amd64 liberror-perl all 0.17029-2 [25.6 kB] Get:23 http://archive.ubuntu.com/ubuntu noble/main amd64 git-man all 1:2.43.0-1ubuntu7 [1098 kB] Get:24 http://archive.ubuntu.com/ubuntu noble/main amd64 git amd64 1:2.43.0-1ubuntu7 [3674 kB] debconf: delaying package configuration, since apt-utils is not installed Fetched 15.8 MB in 0s (36.9 MB/s) Selecting previously unselected package perl-modules-5.38. (Reading database ... 4368 files and directories currently installed.) Preparing to unpack .../00-perl-modules-5.38_5.38.2-3.2build2_all.deb ... Unpacking perl-modules-5.38 (5.38.2-3.2build2) ... Selecting previously unselected package libdb5.3t64:amd64. Preparing to unpack .../01-libdb5.3t64_5.3.28+dfsg2-7_amd64.deb ... Unpacking libdb5.3t64:amd64 (5.3.28+dfsg2-7) ... Selecting previously unselected package libgdbm6t64:amd64. Preparing to unpack .../02-libgdbm6t64_1.23-5.1build1_amd64.deb ... Unpacking libgdbm6t64:amd64 (1.23-5.1build1) ... Selecting previously unselected package libgdbm-compat4t64:amd64. Preparing to unpack .../03-libgdbm-compat4t64_1.23-5.1build1_amd64.deb ... Unpacking libgdbm-compat4t64:amd64 (1.23-5.1build1) ... Selecting previously unselected package libperl5.38t64:amd64. Preparing to unpack .../04-libperl5.38t64_5.38.2-3.2build2_amd64.deb ... Unpacking libperl5.38t64:amd64 (5.38.2-3.2build2) ... Selecting previously unselected package perl. Preparing to unpack .../05-perl_5.38.2-3.2build2_amd64.deb ... Unpacking perl (5.38.2-3.2build2) ... Selecting previously unselected package libexpat1:amd64. Preparing to unpack .../06-libexpat1_2.6.1-2build1_amd64.deb ... Unpacking libexpat1:amd64 (2.6.1-2build1) ... Selecting previously unselected package libkrb5support0:amd64. Preparing to unpack .../07-libkrb5support0_1.20.1-6ubuntu2_amd64.deb ... Unpacking libkrb5support0:amd64 (1.20.1-6ubuntu2) ... Selecting previously unselected package libk5crypto3:amd64. Preparing to unpack .../08-libk5crypto3_1.20.1-6ubuntu2_amd64.deb ... Unpacking libk5crypto3:amd64 (1.20.1-6ubuntu2) ... Selecting previously unselected package libkeyutils1:amd64. Preparing to unpack .../09-libkeyutils1_1.6.3-3build1_amd64.deb ... Unpacking libkeyutils1:amd64 (1.6.3-3build1) ... Selecting previously unselected package libkrb5-3:amd64. Preparing to unpack .../10-libkrb5-3_1.20.1-6ubuntu2_amd64.deb ... Unpacking libkrb5-3:amd64 (1.20.1-6ubuntu2) ... Selecting previously unselected package libgssapi-krb5-2:amd64. Preparing to unpack .../11-libgssapi-krb5-2_1.20.1-6ubuntu2_amd64.deb ... Unpacking libgssapi-krb5-2:amd64 (1.20.1-6ubuntu2) ... Selecting previously unselected package libnghttp2-14:amd64. Preparing to unpack .../12-libnghttp2-14_1.59.0-1build4_amd64.deb ... Unpacking libnghttp2-14:amd64 (1.59.0-1build4) ... Selecting previously unselected package libpsl5t64:amd64. Preparing to unpack .../13-libpsl5t64_0.21.2-1.1build1_amd64.deb ... Unpacking libpsl5t64:amd64 (0.21.2-1.1build1) ... Selecting previously unselected package libbrotli1:amd64. Preparing to unpack .../14-libbrotli1_1.1.0-2build2_amd64.deb ... Unpacking libbrotli1:amd64 (1.1.0-2build2) ... Selecting previously unselected package libsasl2-modules-db:amd64. Preparing to unpack .../15-libsasl2-modules-db_2.1.28+dfsg1-5ubuntu3_amd64.deb ... Unpacking libsasl2-modules-db:amd64 (2.1.28+dfsg1-5ubuntu3) ... Selecting previously unselected package libsasl2-2:amd64. Preparing to unpack .../16-libsasl2-2_2.1.28+dfsg1-5ubuntu3_amd64.deb ... Unpacking libsasl2-2:amd64 (2.1.28+dfsg1-5ubuntu3) ... Selecting previously unselected package libldap2:amd64. Preparing to unpack .../17-libldap2_2.6.7+dfsg-1~exp1ubuntu8_amd64.deb ... Unpacking libldap2:amd64 (2.6.7+dfsg-1~exp1ubuntu8) ... Selecting previously unselected package librtmp1:amd64. Preparing to unpack .../18-librtmp1_2.4+20151223.gitfa8646d.1-2build7_amd64.deb ... Unpacking librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2build7) ... Selecting previously unselected package libssh-4:amd64. Preparing to unpack .../19-libssh-4_0.10.6-2build2_amd64.deb ... Unpacking libssh-4:amd64 (0.10.6-2build2) ... Selecting previously unselected package libcurl3t64-gnutls:amd64. Preparing to unpack .../20-libcurl3t64-gnutls_8.5.0-2ubuntu10_amd64.deb ... Unpacking libcurl3t64-gnutls:amd64 (8.5.0-2ubuntu10) ... Selecting previously unselected package liberror-perl. Preparing to unpack .../21-liberror-perl_0.17029-2_all.deb ... Unpacking liberror-perl (0.17029-2) ... Selecting previously unselected package git-man. Preparing to unpack .../22-git-man_1%3a2.43.0-1ubuntu7_all.deb ... Unpacking git-man (1:2.43.0-1ubuntu7) ... Selecting previously unselected package git. Preparing to unpack .../23-git_1%3a2.43.0-1ubuntu7_amd64.deb ... Unpacking git (1:2.43.0-1ubuntu7) ... Setting up libexpat1:amd64 (2.6.1-2build1) ... Setting up libkeyutils1:amd64 (1.6.3-3build1) ... Setting up libgdbm6t64:amd64 (1.23-5.1build1) ... Setting up libgdbm-compat4t64:amd64 (1.23-5.1build1) ... Setting up libbrotli1:amd64 (1.1.0-2build2) ... Setting up libpsl5t64:amd64 (0.21.2-1.1build1) ... Setting up libnghttp2-14:amd64 (1.59.0-1build4) ... Setting up libkrb5support0:amd64 (1.20.1-6ubuntu2) ... Setting up librtmp1:amd64 (2.4+20151223.gitfa8646d.1-2build7) ... Setting up perl-modules-5.38 (5.38.2-3.2build2) ... Setting up libk5crypto3:amd64 (1.20.1-6ubuntu2) ... Setting up libdb5.3t64:amd64 (5.3.28+dfsg2-7) ... Setting up git-man (1:2.43.0-1ubuntu7) ... Setting up libkrb5-3:amd64 (1.20.1-6ubuntu2) ... Setting up libperl5.38t64:amd64 (5.38.2-3.2build2) ... Setting up libsasl2-modules-db:amd64 (2.1.28+dfsg1-5ubuntu3) ... Setting up perl (5.38.2-3.2build2) ... Setting up libgssapi-krb5-2:amd64 (1.20.1-6ubuntu2) ... Setting up libsasl2-2:amd64 (2.1.28+dfsg1-5ubuntu3) ... Setting up libssh-4:amd64 (0.10.6-2build2) ... Setting up libldap2:amd64 (2.6.7+dfsg-1~exp1ubuntu8) ... Setting up libcurl3t64-gnutls:amd64 (8.5.0-2ubuntu10) ... Setting up liberror-perl (0.17029-2) ... Setting up git (1:2.43.0-1ubuntu7) ... Processing triggers for libc-bin (2.39-0ubuntu8) ... $ pixi --version pixi 0.20.1 $ pixi install Project in /xx/xx/xx/xx is ready to use! $ pixi run pre-commit --version × The current system has a mismatching virtual package. The project requires │ '__linux' to be at least version '5.10' but the system has version │ '5.4.109' Cleaning up project directory and file based variables 00:01 ERROR: Job failed: exit code 1 ```
wolfv commented 5 months ago

Just to be sure, do you require a Linux version this high? I.e. you could also modify your pixi system-requirements towards something lower, possibly?

I'll take a look!

mjkanji commented 5 months ago

@wolfv But that's the thing, I haven't set anything in the system-requirements table. Please see the pixi.toml in my first comment.

wolfv commented 5 months ago

Sorry! Also I think I might have misled you a bit - I believe the kernel version is dictated by the host system and not the docker container (ref: https://stackoverflow.com/questions/32841982/how-can-docker-run-distros-with-different-kernels).

Let me try to recreate the issue locally.

mjkanji commented 5 months ago

Sorry! Also I think I might have misled you a bit - I believe the kernel version is dictated by the host system and not the docker container (ref: https://stackoverflow.com/questions/32841982/how-can-docker-run-distros-with-different-kernels).

Aah, that explains it. Because I was using the Pixi image elsewhere and never ran into this issue. Then, I guess it's not a Pixi issue but rather an issue with the GitLab runner.

wolfv commented 5 months ago

Yeah so we are actually setting the default linux version to 5.10 (https://github.com/prefix-dev/pixi/blob/58ab590d2b83f618ba544c497feef6de0c1c0b7e/src/project/virtual_packages.rs#L18-L21)

Do you happen to know what Gitlab runner you are using? You could possibly fix it by setting the linux version ot something even lower in the system-requirements table:

[system-requirements]
linux = 5.0

Could make it work!

mjkanji commented 5 months ago

Thanks! I'll try and see if I can upgrade the Linux kernel in the GitLab runner (or set the version in system-requirements like you suggested).

Also, closing the issue since it's not really emanating from Pixi.