mamba-org / setup-micromamba

GitHub Action to set up micromamba
MIT License
105 stars 16 forks source link

Micromamba environment caching not working with pip #195

Closed NakulK48 closed 10 months ago

NakulK48 commented 10 months ago

With the old provision-with-micromamba action:

image

As you can see, pip recognises that the environment has been restored, with all of its packages (including those installed via pip)

With the new setup-micromamba action:

image

I can't tell whether this is because it's not caching the pip-installed dependencies at all, or because it's not recognising them in the cache.

jonashaag commented 10 months ago

This is by design because we cache the environment before your pip call now.

NakulK48 commented 10 months ago

What's the reason for that change? It's the setup-micromamba step that initializes your Python shell and chooses a directory for libraries to be installed into via pip. The created micromamba env will have all of your pip dependencies installed inside it.

It sounds like the only way to cache the actual installations would be to delve into the internals, find out where micromamba is installing your pip packages, and manually cache that?

jonashaag commented 10 months ago

Yes. If you use Conda/Micromamba you're not supposed to have a lot of pip installation. You should install the packages with Conda.

pavelzw commented 10 months ago

As Jonas already said, this is by design because with conda environments people tend to alter them after the installation. If we would cache the altered version, you could run into problems with reproducibility. With caching at the beginning of the action and not in the post step, you always get a clean conda environment, it doesn't matter whether you used caching or not. If you really want the pip stuff to be cached too, you can disable caching in setup-micromamba and have an extra actions/cache@v4 step before setup-micromamba of the env location.