search

conda-incubator / conda-store

Data science environments, for collaboration. ✨
https://conda.store
BSD 3-Clause "New" or "Revised" License
142 stars 46 forks source link

[ENH] - Isolate .local (i.e. pip) installations for each environment #677

Closed kcpevey closed 9 months ago

kcpevey commented 9 months ago

Feature description

Currently, if I add a pip section to my environment.yml, the packages get added to my private .local space. However, if I have a second environment which does the same thing, THOSE get added to the .local space as well. This creates a situation where the pip packages from BOTH environments are now available in BOTH environments.

The real kicker is that if the second environment required different versions of the pip packages, it goes ahead and updates the .local, leaving the first environment in a broken state.

This happens very much behind the scenes and users have no idea what happened to their first environment since they "didn't touch it".

We should isolate the pip installed packages so that there is no cross-contamination between environments.

Value and/or benefit

Clean, isolated environments that pip can't break and happy users :)

Anything else?

No response

dharhas commented 9 months ago

This is specific to Nebari because in the Nebari usage of conda-store, the conda folders are readonly so pip uses .local instead.

@costrouc suggested that we could use https://pip.pypa.io/en/stable/cli/pip/#cmdoption-require-virtualenv

I assume this would mean we would need to do a double activation. activate conda env and then the virtualenv. This should work from a terminal but not sure how you could do that within Jupyter.

trallard commented 9 months ago

If this is nebari specific - can we move the issue to the nebari repo?

kcpevey commented 9 months ago

Apologies, I had the wrong repo... https://github.com/nebari-dev/nebari/issues/2133