prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
2.91k stars 161 forks source link

Pixi offline upgrading and everything else #1765

Open toiletsandpaper opened 1 month ago

toiletsandpaper commented 1 month ago

Problem description

Hi! Pixi is a great tool, I'm enjoing using it on my personal laptops and other.

But using pixi for serious commercial projects with cybersec limitations is a huge pain. For example, I have disconnected all my GPU clusters from internet access and setted up mirrors and cachers: Sonatype Nexus, Harbor, etc.; so I can work in a controlled environment. And other tools, like docker, podman, pdm, poetry, pip, conda/mamba is ready and can be upgraded with apt/dnf/pipx - pixi is not (self-upgrade requires internet access and https://github.com/prefix-dev/pixi/issues/1764)

I see only two ways to fix this:

  1. Write and open-source your own Nexus-like tool for mirror and cache packages (maybe like panamax for Rust crates)
  2. Make pixi offline-friendly and also distribute it with apt/dnf/pipx/etc.

Anyways, thank you for your work! Great tool!

pavelzw commented 1 month ago

getting pixi into apt and dnf would be great 👍🏻 not sure how the rules for these packaging ecosystems are, though. you can definitely maintain your own packages but it would be better if we managed to get them upstream into the main repos

ruben-arts commented 1 month ago

You can setup a selfhosted conda package mirror with Quetz

I'd be open to get pixi on pypi but currently don't have the capacity. apt or dnf would be even harder for me.

That said contributions are always welcome! So if anyone has the time and knowledge please help us get pixi distributed though other ecosystems.

toiletsandpaper commented 1 month ago

You can setup a selfhosted conda package mirror with Quetz

I have a problem with this. You can check my other opened issue, that I've linked in topic message, I've explained everything there.

tdejager commented 4 weeks ago

@toiletsandpaper would you also require like an --offline mode, where it does not touch the internet during pixi operations as well?

ruben-arts commented 4 weeks ago

I have a problem with this. You can check my other opened issue, that I've linked in topic message, I've explained everything there.

This issue in #1764 is unrelated to the self-hosted conda package mirror. Let me explain that issue in #1764.

toiletsandpaper commented 4 weeks ago

@toiletsandpaper would you also require like an --offline mode, where it does not touch the internet during pixi operations as well?

What kind of operation requires internet connection? I don't really think that tools like pixi really should require internet at all. Maybe for self-update or initial installation, but nothing else.

I saw pixi trying to dowload pypi-mappings from github. If you want to keep it up to date even if user don't want to upgrade pixi - yeah, --offline mode that forces pixi to disable mapping updating would be great. Maybe even as a flag in .config folder?

tdejager commented 4 weeks ago

Well, a lot of people like the option of having up to date packages and index data by default 😄 This is how most package managers operate. However, thats why uv and other tools offer a --offline flag.

toiletsandpaper commented 4 weeks ago

Well, a lot of people like the option of having up to date packages and index data by default 😄 This is how most package managers operate. However, thats why uv and other tools offer a --offline flag.

Yeah, I get it, but I have pypi and conda mirrors and --offline will ignore my mirrors too, right? If so, then --offline is not that relevant in my case, idk about others :)

But ability to export all packages, move it to offline and set it up with --offline flag looks like a great idea. Sounds like a https://github.com/pdm-project/pdm-download

ruben-arts commented 4 weeks ago

If you only have conda dependencies you can use pixi-pack which is the equivalent of conda-pack.

toiletsandpaper commented 3 weeks ago

I do really hope that someone with good rust expirience can make use of https://github.com/PyO3/maturin that was built to publish rust tools on PyPI. The closes example is https://github.com/astral-sh/uv as it can be downloaded via pip (pipx install uv)