Global environments seem not to be fully separated from each other

fangohr commented 2 months ago

Checks

[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

FROM ghcr.io/prefix-dev/pixi:latest
RUN cd /root
RUN pixi global install ipython cowpy
RUN pixi global install pytest

# note: cowpy can be imported from ipython, but pytest cannot.
# Observation so far: packages that have been installed in the same `pixi global install ` command can import each other
# and are thus not truly in different environments:

RUN echo "This is not meant to work:"
RUN /root/.pixi/bin/ipython -c "import cowpy"
RUN echo "This is okay to fail:"
RUN /root/.pixi/bin/ipython -c "import pytest"

Build the docker image by putting the above into a Dockerfile and run in the same directory docker build -t issuepixi ..

Issue description

As shown in the Dockerfile (as a clean environment we trust), it seems that the different .pixi/venvs can somehow interfere which each other.

My hypothesis is that this connection between the different environments exists when they are installed together in a single pixi global install command.

Expected behavior

In the container above, ipython should not be able to import cowpy if they are truly separated.

fangohr commented 2 months ago

First brought up at https://discord.com/channels/1082332781146800168/1212673297347518464/1278619806240870431 .

tdejager commented 2 months ago

While investigating it seems that cowpy is placed in the ipython site-packages.

baszalmstra commented 2 months ago

I've quickly investigated and found the source of the issue. Whats actually (incorrectly) happening is that when you run pixi global install a b you get two environments, one for a, and one for b. But both environments will contain the same packages!

However, it also looks like some people currently rely on this behavior...

But we are working on a complete revamp of pixi global that will not exhibit this behavior so to not potentially break peoples workflow twice we'll wait for the revamp instead of providing a fix now.

The "solution" for this issue is for now to call pixi global install for each individual spec instead of combining:

pixi global install ipython
pixi global install cowpy

instead of

pixi global install ipython cowpy

fangohr commented 2 months ago

Hi @baszalmstra - thanks for investigating.

Ironically, I was also hoping that the current behaviour is the intended behaviour. (I was looking for a way to provide a global anaconda-like installation which I can recommend to students, and where we can ignore the complexity of environments etc for the first few weeks of learning Python.) Only when looking at the documentation and double checking with @ruben-arts and @wolfv at EuroScipy, I thought this might be bug and the team should be aware of it.

As long as this is on the radar of the pixi team, and we have the issue to document it, that's good. Feel free to close or keep open the ticket - whatever works best for your work flow. From my side, the reporting of the behaviour is completed.

baszalmstra commented 2 months ago

The revamp will have logic where you can specify multiple packages for a single environment so I think that your use case should be covered. Information about the new design can be found here: https://pixi.sh/dev/design_proposals/pixi_global_manifest/

Hofer-Julian commented 1 month ago

The new pixi global in the v0.33.0 release fixes this.

prefix-dev / pixi