anaconda / nb_conda_kernels

Package for managing conda environment-based kernels inside of Jupyter
BSD 3-Clause "New" or "Revised" License
601 stars 70 forks source link

Compatibility with Mamba & Micromamba #234

Open epassaro opened 9 months ago

epassaro commented 9 months ago

Hi! I noticed this extension does not work with micromamba.

My workflow consists of defining alias conda=micromamba and suits almost all my needs.

After looking at manager.py I noticed that the output of micromamba info --json is quite different than the expected one (a lot of different and missing keys).

So my questions are:

  1. Is there a known workaround to make this extension work with micromamba?
  2. Should I write a patch for this?

Thanks

ken-lauer commented 8 months ago

I have a proof-of-concept here that seems to be working in my deployment, at least: https://github.com/ken-lauer/nb_conda_kernels/tree/enh_micromamba I expect there would be a fair amount more work to make it acceptable to the maintainers - were micromamba support a thing they would even consider, that is.

mcg1969 commented 8 months ago

You might be surprised to hear this but I have no problem with the idea of making this work with micromamba. But I do think it is essential that it be well tested.

AniAggarwal commented 8 months ago

Haven't done much development on neither conda nor mamba but I'm happy to help test or contribute; lmk if you think there is something I can help with.

mcg1969 commented 8 months ago

Here's what I think would be required.

mcg1969 commented 8 months ago

Keep in mind that nb_conda_kernels deliberately does not assume that it lives in the same environment as conda; instead, it relies on calling it via a subprocess. This assumption needs to be preserved for mamba as well. That way, you can run Jupyter (lab|notebook) from a child environment and it still works properly.

genghisun commented 6 months ago

Any progress?

I tried @ken-lauer 's repo but it didn't work, I don't know if I'm doing something wrong. Here are my steps:

# in micromamba env with jupyter
> git clone https://github.com/ken-lauer/nb_conda_kernels.git
> cd nb_conda_kernels/
> git checkout enh_micromamba
> pip install -e .
> python -m nb_conda_kernels.install --enable
Enabling nb_conda_kernels...
CONDA_PREFIX: /path/to/micromamba/envs/jupyter
Status: enabled
> jupyter lab
[E 2024-05-16 14:33:09.968 ServerApp] [nb_conda_kernels] couldn't call conda:
    [Errno 2] No such file or directory: 'conda'
[I 2024-05-16 14:33:10.004 ServerApp] [nb_conda_kernels] enabled, 0 kernels found
# and the kernels of other micromamba environments doesn't show up in jupyter lab webui

> python -m nb_conda_kernels list
[ListKernelSpecs] ERROR | [nb_conda_kernels] couldn't call conda:
[Errno 2] No such file or directory: 'conda'
[ListKernelSpecs] [nb_conda_kernels] enabled, 0 kernels found
Available kernels:
  python3    /path/to/micromamba/envs/jupyter/share/jupyter/kernels/python3
> jupyter kernelspec list
Available kernels:
  python3    /path/to/micromamba/envs/jupyter/share/jupyter/kernels/python3
mcg1969 commented 6 months ago

I am not seeing any work being done on this. External contribution will be required. If nb_conda_kernels is essential for you, you'll need to use conda (with the libmamba solver, of course)!

ken-lauer commented 6 months ago

@genghisun I have conda aliased to micromamba for users on our machines as they're used to typing it. The branch linked above, for better or worse, assumes the same is true. A soft-link in your PATH of conda -> micromamba or just having the binary named conda should work.

I'm well aware that the above wouldn't be acceptable for inclusion into anaconda/nb_conda_kernels - but it's still for my own purposes at the moment. It is working well enough on my HPC cluster but is untested elsewhere.

genghisun commented 6 months ago

@ken-lauer Thanks! I made it after ln -s micromamba .local/bin/conda