search

2i2c-org / infrastructure

Infrastructure for configuring and deploying our community JupyterHubs.
https://infrastructure.2i2c.org
BSD 3-Clause "New" or "Revised" License
105 stars 64 forks source link

Openscapes Hub - deploying a new repository with new python 3.9 environment #562

Closed amfriesz closed 2 years ago

amfriesz commented 3 years ago

I'm using nbgitpuller to generate the link to deploy (is that the right term?) a new repository on the Openscapes hub. This new repository has a new environment that is using Python 3.9 and has additional packages relative to the original HLS repository.

The repository I'm try to deploy is here: https://github.com/amfriesz/lpdaac_cloud_data_access.

The completed nbgitpuller link generator form looks like this: image

The link generated is: https://openscapes.2i2c.cloud/hub/user-redirect/git-pull?repo=https%3A%2F%2Fgithub.com%2Famfriesz%2Flpdaac_cloud_data_access.git&urlpath=lab%2Ftree%2Flpdaac_cloud_data_access.git%2F&branch=main

The content from the repository is available from the link, but the environment is running python 3.7 and doesn't contain some of the required python packages found in the environment.yml file (e.g. boto3 and stackstac).

Is there a way to deploy a new environment, specified in the environment.yml, on the Openscapes hub? If so, could I get some guidance on how to do so?

choldgraf commented 3 years ago

Hey @amfriesz - right now there is one environment for the whole hub...are you asking if it's possible to define different environments for each nbgitpuller link? Or do you just imagine updating the hub so that it works with the link you mention?

amfriesz commented 3 years ago

I was thinking about defining different environment for each nbgitpuller link, but updating the hub would work too. Can the hub have multiple environments or kernels available? Say, a py 3.7 and a py 3.9 environment?

choldgraf commented 3 years ago

Hmmm I'm not sure right now whether it's possible to install multiple environments per hub. Will look into that. One option you could explore is having users install the environment during their session directly from the environment.yml files. This usually won't take too long and is just a step people have to run in, e.g., the first cell of a notebook.

amfriesz commented 3 years ago

I did try to create a new environment in the notebook (and in the terminal). The env gets created, but the kernel is not available to run even after I activate the env.

Create the environment image

Activate new environment image

py39 Kernel not available image

choldgraf commented 3 years ago

Hmmmm - that's a great point, I'm not sure how to install and activate a new conda environment interactively without restarting JupyterLab. Lemme ask the team and see what they think, I've also asked around on Twitter to see if others have done this. https://twitter.com/choldgraf/status/1420786716542132224

Is it really important to have both versions of Python available? Would one option be to just upgrade to 3.9?

amfriesz commented 3 years ago

@choldgraf, since the hub is not being heavily used at this time, I don't think upgrading to 3.9 would hurt anything. Also, during our last Openscapes tag-up it was brought to my attention that the following issue is open: https://github.com/2i2c-org/pilot/issues/81. I think it is related to this discussion we're having.

abkfenris commented 3 years ago

We use nb_conda_kernels to allow user managed conda environments on the OceanHackWeek hub. It takes a min for the new kernels to show up, but it doesn't require a restart of JupyterLab.

choldgraf commented 2 years ago

I believe that this is now resolved - closing this but please reopen if there is more to do