search

hackalog / easydata

A flexible template for doing reproducible data science in Python.
MIT License
105 stars 22 forks source link

Is the requirement for nb_conda_kernels still relevant? #227

Open ichitaka opened 3 years ago

ichitaka commented 3 years ago

While I had similar issues as reported in nb_conda_kernels issue #158 some time ago, I can't remember this issue arising since quite some time. Instead, when I try to use this template, I get environment conflicts when calling "make create_environment" which get solved by simply removing the requirement for this old nb_conda_kernels version.

hackalog commented 3 years ago

You're right. It's probably worth taking it out for now and seeing what happens. Since it almost certainly only affects users who used particular nb_conda_kernal versions along the way, the number of people that are potentially still affected is likely quite small

hackalog commented 3 years ago

Incidently, @ichitaka you may want to give the Easydata2 branch a spin, as we're planning a major API break (and many, many cleanups and features). see #225

hamelin commented 1 year ago

Another question around the usefulness of the nb_conda_kernels package comes from the notion that its inclusion in EasyData is backwards. The point of this package is to automate, for a Jupyter deployment, the wrapping of any environment suitable local Conda environment. Thus, the environment in which Jupyter is installed should be where nb_conda_kernels is also installed.

However, EasyData is used in either one of two situations:

  1. Jupyter is run directly from the EasyData project. Then, the Python kernel that is deployed as part of the project's environment is used as that Jupyter's default.
  2. Jupyter is run outside of the EasyData project (as is the case for a shared host interfaced through JupyterHub, for instance). In that case, the wrapping of the EasyData environment into a kernel that can be used through that "external" Jupyter instance depends on it having the nb_conda_kernels installed. If it does not, then the EasyData environment will remain invisible until a kernel wrapping it is explicitly put in place (such as by running python -m ipykernel install ...).

Thus, the inclusion of nb_conda_kernels is either spurious or wrong.

hackalog commented 1 year ago

This is all true. It ended up here a long time ago as part of a corner case where ipykernel was failing to link to the local conda environments when a particular version of conda was in use. As of a year ago, users still occasionally report that the environments aren’t visible from a locally run jupyter notebook.

Note, it does get invoked from both outside and inside the created environment, (create and update_environment respectively).

Since I never finished debugging the issue, and as this step fixed it once, I never took it out.

So I think the right thing to do is fix it in easydata proper, and if/when the bug occurs again, try and solve it properly.

(I’m looking for the issue I filed upstream in ipykernel with the actual problem I was working around. Will post it here if I find it.

hackalog commented 1 year ago

Ah, It wasn’t an issue with ipykernel. It must have been even older than that (before i was using it). I was thinking of this issue:

https://github.com/Anaconda-Platform/nb_conda_kernels/issues/158

Will look at this with an unfoggy head some day soon.