search

anaconda / nb_conda_kernels

Package for managing conda environment-based kernels inside of Jupyter
BSD 3-Clause "New" or "Revised" License
601 stars 70 forks source link

{environment} naming: not clear what's going on #214

Closed nick-youngblut closed 2 years ago

nick-youngblut commented 2 years ago

I have kernels in various project directories. For example:

/ebio/abt3_projects/project1/envs/conda_env1
/ebio/abt3_projects/project1/envs/conda_env2
/ebio/abt3_projects/project2/envs/conda_env1
/ebio/abt3_projects/project2/envs/conda_env2

All of these conda envs include nb_conda_kernels

The default for kernel naming is '{language} [conda env:{environment}]', and so Jupyter (with nb_conda_kernels) lists all of the python kernels for these conda envs as:

Python [conda_env:project1-conda_env1]
Python [conda_env:project1-conda_env2]
Python [conda_env:project2-conda_env1]
Python [conda_env:project2-conda_env2]

So it automatically seems to know the project name. This is not true when I include a different file system:

/ebio/abt3_projects/project1/envs/conda_env1
/ebio/abt3_projects/project1/envs/conda_env2
/ebio/abt3_projects/project2/envs/conda_env1
/ebio/abt3_projects/project2/envs/conda_env2
/tmp/global2/project3/envs/conda_env1
/tmp/global2/project3/envs/conda_env2

Which results in no automatic project naming for the kernel :

Python [conda_env:project1-conda_env1]
Python [conda_env:project1-conda_env2]
Python [conda_env:project2-conda_env1]
Python [conda_env:project2-conda_env2]
Python [conda_env:conda_env1]
Python [conda_env:conda_env2]

There is no {environment} for the /tmp/global2/ conda envs.

Which leads to problems with multiple conda envs of the same name, but different locations:

/ebio/abt3_projects/project1/envs/conda_env1
/ebio/abt3_projects/project1/envs/conda_env2
/ebio/abt3_projects/project2/envs/conda_env1
/ebio/abt3_projects/project2/envs/conda_env2
/tmp/global2/project3/envs/conda_env1
/tmp/global2/project3/envs/conda_env2
/tmp/global2/project4/envs/conda_env1
/tmp/global2/project4/envs/conda_env2

... produces:

Python [conda_env:project1-conda_env1]
Python [conda_env:project1-conda_env2]
Python [conda_env:project2-conda_env1]
Python [conda_env:project2-conda_env2]
Python [conda_env:conda_env1]
Python [conda_env:conda_env2]
Python [conda_env:conda_env1-2]
Python [conda_env:conda_env2-2]

Any idea why I'm seeing this behavior?

nick-youngblut commented 2 years ago

I figured out the naming, which most people likely won't run into. The {environment} naming comes from running jupyter notebook servers in each of those directories. I run one per project. I don't have any notebooks running in the /tmp/global2/ projects, which is why the {environment} is missing for those kernels.

I'm guessing that there is no way to change the naming in this case. Due to inode limitations for our projects file system, we need to migrate all of our conda envs to another file system, so the conda envs will be in a different location than the running jupyter notebooks, and hence the naming issue.

I guess that we could just name all conda envs with the project as the prefix (i.e., conda env name: {project}-{conda_env}, but this can result in long conda env names.