prefix-dev / pixi

Package management made easy
https://pixi.sh
BSD 3-Clause "New" or "Revised" License
3.39k stars 193 forks source link

Make pixi work without internet connection #2555

Open certik opened 2 days ago

certik commented 2 days ago

When I am not connected to the internet, I get:

$ pixi r build
ERROR error=error sending request for url (https://conda.anaconda.org/conda-forge/osx-arm64/repodata.json.zst)
ERROR error=error sending request for url (https://conda.anaconda.org/conda-forge/osx-arm64/repodata.json.zst)
  ⠁ build:win-64         [00:00:00] loading repodata
  ⠁ build:linux-64       [00:00:00] loading repodata
  ⠁ host:osx-arm64       [00:00:00] loading repodata
  ⠁ host:win-64          [00:00:00] loading repodata
  ⠁ host:linux-64        [00:00:00] loading repodata
  ⠁ default:osx-arm64    [00:00:00] loading repodata
  ⠁ default:linux-64     [00:00:00] loading repodata
  ⠁ default:win-64       [00:00:00] loading repodata                              × error sending request for url (https://conda.anaconda.org/conda-forge/osx-
  │ arm64/repodata.json.zst)
  ├─▶ client error (Connect)
  ├─▶ dns error: failed to lookup address information: nodename nor servname
  │   provided, or not known
  ╰─▶ failed to lookup address information: nodename nor servname provided, or
      not known

This is in contrast to conda where I just do "conda activate lf" and I can work without an internet.

Part of this issue is that pixi stores all the environments in .pixi in the current directory, so if I do "git clean -dfx" I lose it.

However, it would be great if pixi could simply used the already downloaded packages from ~/.pixi and rebuild the environment without the internet. Then I can easily work with pixi and different environments and creating environments while offline.

Hofer-Julian commented 1 day ago

I just tried the following:

pixi init test-offline
cd test-offline
pixi add python
pixi run python
# cut internet connection
rm -rf .pixi
pixi run python

This just worked for me. Can you please provide a reproducer for your issue?

wolfv commented 1 day ago

@Hofer-Julian I think this happens when the .pixi folder is removed. We could still use the cached packages.

@certik do you have the lockfile still?

Hofer-Julian commented 1 day ago

@Hofer-Julian I think this happens when the .pixi folder is removed.

That's what I did :)

certik commented 1 day ago

I don't have a lock file. Here is what I did yesterday:

$ git clone https://github.com/lfortran/lfortran
$ cd lfortran
$ git checkout fbcdd5546be8631cabc97c2ebfa17bb024ba9526
$ pixi r build

Then disconnect from the internet:

$ git clean -dfx
$ pixi r build

However there was some time between the two parts. I just did this again in succession and it works! (I did this twice now, it clearly works.)

But above I posted what happened when I tried to build yesterday, and it clearly fails.

When does pixi require internet access? Is there some timeout after which pixi decides to redownload the repodata? That would explain what I've seen. In that case a solution is to add an option to skip redownloading and use the (old) repodata.

As a user, I would like to pre-download whatever pixi needs ahead of time, so that I can work without internet access locally. Pixi seems to have a very usable local cache in ~/Library/Caches/rattler/cache that clearly worked just now for me, even when I removed the lfortran/.pixi and a lock file. So it seems it is 99% there already.

certik commented 1 day ago

(The following Pixi task clean = "git clean -dfx -e '.pixi'" takes care of cleaning up my local repository except .pixi, which greatly speeds up builds.)