Additional packages for module 1 ESPM-157

[cboettig]

Package Name

altair, ibis-framework[duckdb], leafmap[libremap], myst

Hub URL

nature.datahub.berkeley.edu

Course Name

ESPM 157

Semester Details

Fall 2024

Installation Deadline

Start of Semester.

Let me know if it would be better to do separate requests? Also, I know most of these we can install ourselves but it looks like myst will require a working nodejs setup.

[balajialg]

@cboettig Added all the requested packages in https://staging.datahub.berkeley.edu/ via this PR1 & PR2. I currently tested this notebook from ESPM 157 Github repo. It seems like ibis-framework[pandas] is the package required to get the notebook to execute successfully. Do you also need the ibis-framework package which has a large scope in addition to ibis-framework[pandas] - https://ibis-project.org/install

With ibis-framework & without ibis-framework[pandas], I get the following error.

[balajialg]

Closing this issue as changes got merged to prod. @cboettig Please reopen this issue if that is not the case.

[cboettig]

@balajialg thanks much! Apologies I'm looping back here and catching up. Yes, can we have both pandas and duckdb backends for ibis? ibis-framework[pandas,duckdb] . (We may be able to stick with pure duckdb if https://github.com/duckdb/duckdb/issues/13121 is merged soon :grin: )

I don't think I have permissions re-open.

• [x] Also maybe should be a separate request, but would it be possible to have a 4GB allocation for all ESPM-157 students? Testing right now, on my allocation I see just 1 GB on the instance. We can probably make that work if necessary, but it is rather slim...
• [ ] I see we have code-server proxy on datahub.berkeley.edu, but the jupyter extensions aren't installed. Would it be possible to add these? I try adding manually but get an error about version conflict with the code-server version currently on the hub.

[balajialg]

@cboettig Sure, I am assuming that you would want https://pypi.org/project/jupyter-vscode-server/ extension. Any other code server extension that you want as part of the image?

For increasing RAM, we need the bcourses id for your FA 24 course. Can you either share the bcourses id here or create a separate request via this github issue. I will create a PR to address all the above requests.

[cboettig]

@balajialg Thanks much!! It looks like jupyter-vscode-server already installed on datahub.berkeley.edu. However, Code-server itself has it's own extension system, but it can't install:

for whatever reason, code-server also doesn't detect a python installation (probably something about how conda is configured?). It would be great if users could could go to e.g. https://datahub.berkeley.edu/user/cboettig/vscode/ (or appropriate link) and be able to start running in an ipynb notebook there.

[balajialg]

Thanks @cboettig, will look into it.

[cboettig]

@balajialg One more python module request for you. (Sorry, just finding so much awesome stuff in the python ecosystem while migrating from the R version of this course. I know we could do quarto but Myst is :heart_eyes_cat: )

• [ ] install jupyterlab_myst

[balajialg]

@cboettig No worries, keep it coming :P

[ryanlovett]

@cboettig Do you mean pre-install the Jupyter and Python extensions?

@balajialg I think this can be done by adding something like this to the docker image:

export VSCODE_EXTENSIONS=${CONDA_DIR}/share/code-server/extensions
code-server --extensions-dir ${VSCODE_EXTENSIONS} --install-extension ms-python.python --force
code-server --extensions-dir ${VSCODE_EXTENSIONS} --install-extension ms-toolsai.jupyter

To be reproducible, we'd need to first download specific versions and then install, e.g.

export VSCODE_EXTENSIONS=${CONDA_DIR}/share/code-server/extensions

VSIX=2024.13.2024080701
wget https://marketplace.visualstudio.com/_apis/public/gallery/publishers/ms-python/vsextensions/python/${VSIX}/vspackage -O /tmp/ms-python.vsix
code-server --extensions-dir ${VSCODE_EXTENSIONS} --install-extension /tmp/ms-python.vsix 

[balajialg]

Thanks @ryanlovett! This is helpful instructions. I will get to this asap!

[balajialg]

@ryanlovett Just created a PR - https://github.com/berkeley-dsep-infra/datahub/pull/5915 which addresses Carl's requirement

[balajialg]

@cboettig Just wanted to give you a heads up about where things are currently - We are seeing 3 different challenges to get the extension set up working in VSCode as expected before the start of the instruction,

a) Downloading VSIX files for the required VSCode extensions via wget/curl is failing since there is a discrepancy in the size of the VSIX files when downloaded via GUI vs CLI. We might need to download it via UI, place it in datahub directory and reference it in the docker image. Ref: https://github.com/microsoft/vscode-python/issues/17491 b) Ensuring that a single Python kernel is selected in the VSCode interface amongst multiple kernels requires 3rd party extensions which are not available in the code-server aka open-source version of VSCode. Ref: https://coder.com/docs/code-server/FAQ#how-does-the-config-file-work. We are looking at other options to circumvent this. c) Hiding other kernels so that a default kernel is selected needs to be done on a per-user/workspace basis and from the limited information we have currently, it might be hard to change default settings. Ref: https://github.com/microsoft/vscode-jupyter/discussions/13032#discussioncomment-8049433

Jonathan, Ryan and I met yesterday for a debugging session to get this to work (Jonathan and I had a marathon meeting lasting 3+ hours). We will be meeting again next week to hash out the next steps.

Being transparent - If we cannot get it to work the way you wanted during the next meeting, we might need to postpone handling this request by a month or so since the semester's start is a really hectic time to deep dive into a single issue.

[cboettig]

ok thanks for the heads up. I think we'll try and stick with pure jupyterhub interface then

[cboettig]

@balajialg @ryanlovett

Hopefully quick/small thing here. I'd like students to be able to delete folders/directories even if they aren't empty. (I find most GUIs allow this, and not being able to delete non-empty dirs confuses students.) I gather it's possible but couldn't figure out the configuration option (see https://discourse.jupyter.org/t/recursive-delete-directories-in-file-browser/27468/2) any ideas? Is this something we could enable out of the box?

(I know this is potentially destructive, but I really emphasize that students should treat hub storage as ephemeral anyway and want them to be good stewards of that space cleaning up storage without teaching bash commands)

[felder]

@cboettig one concern that'd I'd have is we're currently talking about the datahub hub, so a config change for one class will impact all others on the same hub. I wholeheartedly support the idea of being a good steward of resources and informing students to consider hub storage as ephemeral including encouraging folks do their own backups. I wish more people did this!

However, as you've indicated this is potentially destructive and represents a departure from how non empty directories are currently treated. It could present a problem for other courses where less focus is spent talking about resource management. What I'd love to see here is an upstream change where a recursive delete requires a confirmation perhaps even with a checkbox to acknowledge when one is deleting a non empty directory. That would allow for this functionality with less of a chance of a destructive mistake.

[ryanlovett]

I think it is possible to mount code containing a FileContentsManager.always_delete_dir setting to the jupyter config directory just for a specific course. While we do something like this in hub/values.yml:

singleuser:
  extraFiles:
    jupyter_server_config.json:
      mountPath: /usr/local/etc/jupyter/jupyter_server_config.json
      data:
        FileContentsManager:
          always_delete_dir: true

it would need to be ported to kubespawner override form in order to take effect for just the classes we want. I think it would need to be a volumeMount, e.g.

volumeMounts:
- name: home
  subPath: _espm-157/espm-157.json
  mountPath: /usr/local/etc/jupyter/jupyter_server_config.d/espm-157.json

where we put the config in the filestore in a course subdirectory. We do this for shared directories, but instead of sharing data, we'd "share" configuration. The contents of the file would be:

{"FileContentsManager":{"always_delete_dir":true}}

We may or may not need to precreate /usr/local/etc/jupyter/jupyter_server_config.d. (I don't know if kubernetes would make that for us when it performs the mount.)

See /srv/conda/etc/jupyter/jupyter_server_config.d for config files installed by packages.

Note that I'd rather that this config be code deployed from the repo rather than from a directory on the filestore. One way to do that would be to put the configuration data inside a configMap. There's no existing example of that, but it may not be too hard. The above should work though.

[cboettig]

@balajialg for some reason import seaborn seems broken for me on the hub? I get this trace:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[1], line 2
      1 import ibis
----> 2 import seaborn as sns

File /srv/conda/lib/python3.11/site-packages/seaborn/__init__.py:9
      7 from .categorical import *  # noqa: F401,F403
      8 from .distributions import *  # noqa: F401,F403
----> 9 from .matrix import *  # noqa: F401,F403
     10 from .miscplot import *  # noqa: F401,F403
     11 from .axisgrid import *  # noqa: F401,F403

File /srv/conda/lib/python3.11/site-packages/seaborn/matrix.py:12
      9 import pandas as pd
     10 from scipy.cluster import hierarchy
---> 12 from . import cm
     13 from .axisgrid import Grid
     14 from .utils import (
     15     despine,
     16     axis_ticklabels_overlap,
   (...)
     19     _draw_figure,
     20 )

File /srv/conda/lib/python3.11/site-packages/seaborn/cm.py:1582
   1579     _cmap_r = colors.ListedColormap(_lut[::-1], _name + "_r")  
   1580     locals()[_name + "_r"] = _cmap_r
-> 1582     mpl_cm.register_cmap(_name, _cmap)
   1583     mpl_cm.register_cmap(_name + "_r", _cmap_r)
   1585 del colors, mpl_cm

AttributeError: module 'matplotlib.cm' has no attribute 'register_cmap'

Also, lemme know if we might get Myst markdown back. Also let me know if you have a reproducible example of the issue, I saw the forum post you've linked and tried asking around a bit and only hear that myst should work well with LaTeX?

[cboettig]

(it looks like I can resolve the seaborn issue by upgrading via conda install seaborn, but would be much preferable to have seaborn working out of the box on day one of class next week... :worried:

[felder]

@cboettig currently seaborn is version locked on datahub:

datahub/images/default/environment.yml:27:- seaborn=0.11.*

When I look:

jovyan@jupyter-felder:~$ mamba list | grep -i seaborn

seaborn                   0.11.2               hd8ed1ab_0    conda-forge
seaborn-base              0.11.2             pyhd8ed1ab_0    conda-forge
jovyan@jupyter-felder:~$ mamba install seaborn

Looking for: ['seaborn']

conda-forge/noarch                                  16.1MB @  29.0MB/s  0.9s
conda-forge/linux-64                                37.1MB @  28.9MB/s  2.4s

Pinned packages:
  - python 3.11.*

Transaction

  Prefix: /srv/conda

  All requested packages already installed

jovyan@jupyter-felder:~$ mamba update seaborn

Looking for: ['seaborn']

conda-forge/linux-64                                        Using cache
conda-forge/noarch                                          Using cache

Pinned packages:
  - python 3.11.*

Transaction

  Prefix: /srv/conda

  Updating specs:

   - seaborn
   - ca-certificates
   - certifi
   - openssl

  Package         Version  Build         Channel          Size
────────────────────────────────────────────────────────────────
  Upgrade:
────────────────────────────────────────────────────────────────

  - seaborn-base   0.11.2  pyhd8ed1ab_0  conda-forge     229kB
  + seaborn-base   0.13.2  pyhd8ed1ab_2  conda-forge     235kB
  - seaborn        0.11.2  hd8ed1ab_0    conda-forge       4kB
  + seaborn        0.13.2  hd8ed1ab_2    conda-forge       7kB

  Summary:

  Upgrade: 2 packages

  Total download: 242kB

────────────────────────────────────────────────────────────────

Confirm changes: [Y/n] y
seaborn                                              7.0kB @  47.6kB/s  0.1s
seaborn-base                                       234.6kB @   1.4MB/s  0.2s

Downloading and Extracting Packages:

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
jovyan@jupyter-felder:~$ 

It is definitely not recommended to have students running conda to update packages manually as those updates will not persist between server launches. Can you please confirm that seaborn 0.13.2 meets your needs (or is the version that gets deployed when you run conda install). If so we can probably adjust the version lock for seaborn to deploy 0.13.2.

Also for the future, we version lock packages so that subsequent deployments during the semester will not result in unintended package changes which have a habit of breaking things. It's really helpful if when specifying packages you also specify versions.

[felder]

I was able to reproduce your issue in datahub prod using a newly created notebook that only includes:

import seaborn

After upgrading to 0.13.2 via mamba on my running pod, I can confirm the import statement does not error. I'll go ahead and do the version bump.

[felder]

@cboettig it'll be a while before this gets deployed. Once the above PR makes it to staging I'll test and assuming seaborn is fixed I'll submit another PR to merge and deploy this change to prod.

[cboettig]

Thanks @felder !

(yup, :100: on board with not having students conda install, and I like that it gets reset between server launches because that's a great way to get back to a working system if students do go about running conda install and getting things stuck. And I appreciate that seaborn is a widely used package on the hub and locked version for good reason. Thanks for confirming the current version is broken for more than just me :-) !)

[balajialg]

Thanks @felder for quickly handling the issue

Also, lemme know if we might get Myst markdown back. Also let me know if you have a reproducible example of the issue, I saw the forum post you've linked and tried asking around a bit and only hear that myst should work well with LaTeX?

@cboettig Here is a notebook shared by the instructor for example: 01_basics. You can see that the Latex output is not compiled in the markdown cell and as a result you can see output like - "Compute the remainder of "$\dfrac{27}{5}$" in Python". You can uninstall JupyterLab-myst package from pypi manager (after opening the notebook in a lab interface) and can observe that the latex output is rendering as expected.

If the instructor used data8.datahub, things render correctly. Here is the same notebook again that renders as expected in the data8 hub which doesn't have the myst package: 01_basics.

We will discuss this internally during a meeting and see if we can find a better approach to get you myst package.

[cboettig]

@balajialg ha, I see what you mean. Myst isn't breaking the LaTeX per se though, the issue is that Myst isn't parsing the MathJax (LaTeX) that it finds inside other HTML elements, like so:

<div class="alert alert-block alert-info"> Compute the remainder of  \dfrac{27}{5} in Python.</div>

[cboettig]

by the way, I believe the Myst way of creating "Admonitions" like the other instructor is using in markdown chunks is to do something like

:::{note}

 **TRY IT!** Compute the remainder of  $\dfrac{27}{5}$ in Python.

but I certainly appreciate they don't want to change scores of markdown cells in all their notebooks by hand right now!

[felder]

@cboettig change for seaborn is live on prod. Restart your server and please confirm.

[cboettig]

@felder :tada: seaborn works out of the box again. thanks!

[balajialg]

@cboettig Thanks, that is helpful! We just had a team meeting and discussed in detail the Myst issue. Few ideas got proposed during the meeting as next steps,

• We want to isolate your course from other courses using vanilla DataHub so that you can customize the hub the way you want. If you can draft a detailed requirement for the new hub (resource, package requirement - Python/R etc..), it will make it easy to get something up and running soon before the start of instruction. I can set up a requirement meeting if you want to collectively brainstorm with us about the feasibility.
• We were thinking of having the hub with the URL cnr.datahub.berkeley.edu (Thanks @ryanlovett for the suggestion) since ESPM courses are affiliated with the College of Natural Resources. I am doing some digging to find other courses from CNR that use the main Datahub which can be ported to the new hub. Let us know if are okay with it.
• Since this is going to be an activity with a steep deadline, Would you be open to creating PRs for this hub and customizing it (aka sharing workload with admins)? That way, admins can also focus on other time-sensitive tasks. Let us know.

On another note, Eric and I will work with Mohammad to see if he can modify the notebooks to be compatible with myst. No guarantees here though.

[ryanlovett]

To follow up on what @balajialg said, we assume you'd prefer a rocker-based image for both python+r environments -- just let us know what the starting point should be, and what apps and libraries you need beyond the recent requests.

[cboettig]

Thanks team!

• This plan sounds fine to me, but we can get by on datahub for the start.
• Resources-wise, you already have the core things I need on datahub -- 4GB ram instances and being able to see hidden files, and support for gh-scoped-creds.
• For ESPM-157 this Fall, my plan is now a pure-python experience. We'll be okay without RStudio/R support, though yeah I'll likely still be figuring out our precise python stack a bit more as we go along.
• Myst would be nice but we can do without it (or I teach them some package management which yeah I might regret but won't be the only ill-advised thing I try this semester anyway :upside_down_face: ).
• We don't need the vscode/code-server environment, that's mostly another experiment of mine that was largely to address issues like the hidden files, gh-scoped-creds, and directory delete that we've more-or-less solved on the hub side now anyway.
• Branding-wise we're technically the "Rausser College of Natural Resources", or RCNR now. Though actually internet wise we've always been "nature.berkeley.edu" as a subdomain, so nature.datahub.berkeley.edu could work too.
• I'm happy to take a whack at my own PRs though I don't know my way around all the config files as well as y'all do. I wouldn't want to breaking stuff for other RCNR faculty!

[balajialg]

Awesome, thanks @cboettig! nature.datahub.berkeley.edu sounds great. Few follow up questions,

• What will be an ideal timeline for the deployment of the new hub? If we get something deployed in the next 2-4 weeks, will that work for you?
• Launching the hub defaults to Jupyter Lab application?
• How much will the storage requirement for your course look like? (any tentative estimate will be of help to provision storage)
• When you mean hidden files, do you refer to this feature?

[cboettig]

@balajialg thanks!

• Our first day of class (and first day using the Hub) is Aug 28th. We can start with module 1 on the standard https://datahub.berkeley.edu though and plan to switch over at module 2 (probably around week 4, https://espm-157.carlboettiger.info/overview/schedule.html). Some students might be confused and keep logging into the older server, but if that's the biggest confusion we have we'll be lucky :-) .
• Yes, default JupyterLab is good.
• Standard storage provision is good (not sure what it is on https://datahub.berkeley.edu, but ~ 5 GB should be fine. Persistent storage could be less, say, 2 GB, but some tmp storage would be great where it can help accelerate certain I/O ops or where we want to compare streaming vs download+read-from-disk workflows.
• yes, hidden file config was already solved in https://github.com/berkeley-dsep-infra/datahub/issues/5825.

Basically everything I've hit so far is already solved except the myst-jupyterlab conflicts. btw myst devs suggested they could fix this upstream, though obviously may take a while yet.

[balajialg]

Thanks @cboettig, that's helpful info! We should target getting the hub ready before the week of Sep 16-20.

[balajialg]

@cboettig - @felder is starting to deploy the new nature hub in prep for your classes during week 4. It would be helpful to know the consolidated list of Python packages that are required for your course. I have listed some base Python packages + the list of packages you had requested historically. Can you please review the list below and let us know of any additions/deletions/edits? I copied the versions from the DataHub image but please specify if you have preference on the exact version for these packages. Appreciate your input!

- numpy=1.26.*
- matplotlib=3.9.*
- scipy=1.10.0
- ipympl=0.9.*
- pandas==2.2.2
- statsmodels=0.13.5
- scikit-learn=1.4.*
- seaborn=0.13.2
- decorator=5.0.*
- networkx=2.6.*
- spacy=3.7.3
- nltk=3.6.*
- ibis-framework[pandas,duckdb]==9.2.0
- jupyter-vscode-proxy==0.6
- code-server==4.23.1
- altair==5.3.0
- leafmap==0.36.4
- mystmd==1.3.1
- jupyterlab-git==0.50.1

[felder]

FYI here's a huge list of packages that are installed:

jovyan@fb175f20ed24:~/.cache$ mamba list
# packages in environment at /srv/conda/envs/notebook:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
aiofiles                  22.1.0             pyhd8ed1ab_0    conda-forge
aiohappyeyeballs          2.4.0              pyhd8ed1ab_0    conda-forge
aiohttp                   3.10.5          py311h61187de_0    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
aiosqlite                 0.19.0             pyhd8ed1ab_0    conda-forge
alembic                   1.12.0             pyhd8ed1ab_0    conda-forge
altair                    5.4.1              pyhd8ed1ab_0    conda-forge
annotated-types           0.7.0              pyhd8ed1ab_0    conda-forge
anyio                     3.7.1              pyhd8ed1ab_0    conda-forge
anywidget                 0.9.13             pyhd8ed1ab_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               23.1.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0          py311h459d7ec_4    conda-forge
arrow                     1.3.0              pyhd8ed1ab_0    conda-forge
asgiref                   3.8.1              pyhd8ed1ab_0    conda-forge
asttokens                 2.4.0              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4              pyhd8ed1ab_0    conda-forge
async_generator           1.10                       py_0    conda-forge
atk-1.0                   2.38.0               h04ea711_2    conda-forge
atpublic                  3.1                pyhd8ed1ab_0    conda-forge
attrs                     23.1.0             pyh71513ae_1    conda-forge
aws-c-auth                0.7.26               hc36b679_2    conda-forge
aws-c-cal                 0.7.4                h2abdd08_0    conda-forge
aws-c-common              0.9.27               h4bc722e_0    conda-forge
aws-c-compression         0.2.19               haa50ccc_0    conda-forge
aws-c-event-stream        0.4.3                h570d160_0    conda-forge
aws-c-http                0.8.8                h9b61739_1    conda-forge
aws-c-io                  0.14.18              h49c7fd3_7    conda-forge
aws-c-mqtt                0.10.4              h5c8269d_18    conda-forge
aws-c-s3                  0.6.4               h77088c0_11    conda-forge
aws-c-sdkutils            0.1.19               h038f3f9_2    conda-forge
aws-checksums             0.1.18              h038f3f9_10    conda-forge
aws-crt-cpp               0.28.0               hf262114_1    conda-forge
aws-sdk-cpp               1.11.379             h2b5e2e4_6    conda-forge
azure-core-cpp            1.13.0               h935415a_0    conda-forge
azure-identity-cpp        1.8.0                hd126650_2    conda-forge
azure-storage-blobs-cpp   12.12.0              hd2e3451_0    conda-forge
azure-storage-common-cpp  12.7.0               h10ac4d7_1    conda-forge
azure-storage-files-datalake-cpp 12.11.0              h325d260_1    conda-forge
babel                     2.13.0             pyhd8ed1ab_0    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.5              pyhd8ed1ab_0    conda-forge
beautifulsoup4            4.12.2             pyha770c72_0    conda-forge
bidict                    0.23.1             pyhd8ed1ab_0    conda-forge
bleach                    6.1.0              pyhd8ed1ab_0    conda-forge
blinker                   1.6.3              pyhd8ed1ab_0    conda-forge
blosc                     1.21.6               hef167b5_0    conda-forge
bqplot                    0.12.43            pyhd8ed1ab_0    conda-forge
branca                    0.7.2              pyhd8ed1ab_0    conda-forge
brotli                    1.1.0                hd590300_1    conda-forge
brotli-bin                1.1.0                hd590300_1    conda-forge
brotli-python             1.1.0           py311hb755f60_1    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.33.1               heb4867d_0    conda-forge
ca-certificates           2024.7.4             hbcca054_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cairo                     1.18.0               hebfffa5_3    conda-forge
certifi                   2024.7.4           pyhd8ed1ab_0    conda-forge
certipy                   0.1.3                      py_0    conda-forge
cffi                      1.16.0          py311hb3a22ac_0    conda-forge
cfitsio                   4.4.1                hf8ad068_0    conda-forge
charset-normalizer        3.3.0              pyhd8ed1ab_0    conda-forge
click                     8.1.7           unix_pyh707e725_0    conda-forge
click-plugins             1.1.1                      py_0    conda-forge
cligj                     0.7.2              pyhd8ed1ab_1    conda-forge
code-server               4.23.1               hafe34db_1    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colour                    0.1.5              pyhd8ed1ab_1    conda-forge
comm                      0.1.4              pyhd8ed1ab_0    conda-forge
configurable-http-proxy   4.6.2                he2f69ee_0    conda-forge
contourpy                 1.2.1           py311h9547e67_0    conda-forge
cryptography              41.0.4          py311h63ff55d_0    conda-forge
cycler                    0.12.1             pyhd8ed1ab_0    conda-forge
debugpy                   1.8.0           py311hb755f60_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.3              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.6.2                h59595ed_0    conda-forge
filelock                  3.15.4             pyhd8ed1ab_0    conda-forge
fiona                     1.9.6           py311h3c39baf_4    conda-forge
fmt                       11.0.2               h434a139_0    conda-forge
folium                    0.17.0             pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_2    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.53.1          py311h61187de_0    conda-forge
fqdn                      1.5.1              pyhd8ed1ab_0    conda-forge
freetype                  2.12.1               h267a509_2    conda-forge
freexl                    2.0.0                h743c826_0    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
frozenlist                1.4.1           py311h459d7ec_0    conda-forge
fsspec                    2024.6.1           pyhff2d567_0    conda-forge
gdal                      3.9.2           py311hb17e472_0    conda-forge
gdk-pixbuf                2.42.12              hb9ae30d_0    conda-forge
gdown                     5.2.0              pyhd8ed1ab_0    conda-forge
geojson                   3.1.0              pyhd8ed1ab_0    conda-forge
geopandas                 0.14.4             pyhd8ed1ab_0    conda-forge
geopandas-base            0.14.4             pyha770c72_0    conda-forge
geos                      3.12.2               he02047a_1    conda-forge
geotiff                   1.7.3                hf7fa9e8_2    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
giflib                    5.2.2                hd590300_0    conda-forge
git                       2.46.0          pl5321hb5640b7_0    conda-forge
glog                      0.7.1                hbabe93e_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
graphite2                 1.3.13            h59595ed_1003    conda-forge
graphviz                  12.0.0               hba01fac_0    conda-forge
greenlet                  3.0.3                    pypi_0    pypi
gtk2                      2.24.33              h6470451_5    conda-forge
gts                       0.7.6                h977cf35_4    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
h2                        4.1.0              pyhd8ed1ab_0    conda-forge
harfbuzz                  9.0.0                hda332d3_1    conda-forge
hdf4                      4.2.15               h2a13503_7    conda-forge
hdf5                      1.14.3          nompi_hdf9ad27_105    conda-forge
hpack                     4.0.0              pyh9f0ad1d_0    conda-forge
htmltools                 0.5.3              pyhd8ed1ab_0    conda-forge
httpcore                  1.0.5              pyhd8ed1ab_0    conda-forge
httpx                     0.27.2             pyhd8ed1ab_0    conda-forge
humanize                  4.10.0             pyhd8ed1ab_0    conda-forge
hyperframe                6.0.1              pyhd8ed1ab_0    conda-forge
ibis-framework            9.3.0                hd8ed1ab_0    conda-forge
ibis-framework-core       9.3.0              pyhd8ed1ab_0    conda-forge
icu                       75.1                 he02047a_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
importlib-metadata        6.8.0              pyha770c72_0    conda-forge
importlib-resources       6.1.0              pyhd8ed1ab_0    conda-forge
importlib_metadata        6.8.0                hd8ed1ab_0    conda-forge
importlib_resources       6.1.0              pyhd8ed1ab_0    conda-forge
ipyevents                 2.0.2              pyhd8ed1ab_0    conda-forge
ipyfilechooser            0.6.0              pyhd8ed1ab_0    conda-forge
ipykernel                 6.25.2             pyh2140261_0    conda-forge
ipyleaflet                0.19.2             pyhd8ed1ab_0    conda-forge
ipysheet                  0.7.0              pyhd8ed1ab_0    conda-forge
ipython                   8.16.1             pyh0d859eb_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipytree                   0.2.2              pyhd8ed1ab_0    conda-forge
ipyvue                    1.11.0             pyhd8ed1ab_0    conda-forge
ipyvuetify                1.10.0             pyhd8ed1ab_0    conda-forge
ipywidgets                8.1.1              pyhd8ed1ab_0    conda-forge
isoduration               20.11.0            pyhd8ed1ab_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
joblib                    1.4.2              pyhd8ed1ab_0    conda-forge
json-c                    0.17                 h1220068_1    conda-forge
json5                     0.9.14             pyhd8ed1ab_0    conda-forge
jsonpointer               2.4             py311h38be061_3    conda-forge
jsonschema                4.19.1             pyhd8ed1ab_0    conda-forge
jsonschema-specifications 2023.7.1           pyhd8ed1ab_0    conda-forge
jsonschema-with-format-nongpl 4.19.1             pyhd8ed1ab_0    conda-forge
jupyter-lsp               2.2.5              pyhd8ed1ab_0    conda-forge
jupyter-offlinenotebook   0.2.2              pyh1d7be83_0    conda-forge
jupyter-resource-usage    1.1.0              pyhd8ed1ab_0    conda-forge
jupyter-server-proxy      4.4.0              pyhd8ed1ab_0    conda-forge
jupyter-vscode-proxy      0.6                pyhd8ed1ab_0    conda-forge
jupyter_client            7.4.9              pyhd8ed1ab_0    conda-forge
jupyter_core              5.4.0           py311h38be061_0    conda-forge
jupyter_events            0.10.0             pyhd8ed1ab_0    conda-forge
jupyter_leaflet           0.19.2             pyhd8ed1ab_0    conda-forge
jupyter_server            2.14.2             pyhd8ed1ab_0    conda-forge
jupyter_server_fileid     0.9.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.5.3              pyhd8ed1ab_0    conda-forge
jupyter_server_ydoc       0.8.0              pyhd8ed1ab_0    conda-forge
jupyter_telemetry         0.1.0              pyhd8ed1ab_1    conda-forge
jupyter_ydoc              0.2.4              pyhd8ed1ab_0    conda-forge
jupyterhub                4.1.6              pyh31011fe_0    conda-forge
jupyterhub-base           4.1.6              pyh31011fe_0    conda-forge
jupyterhub-singleuser     4.1.6              pyh31011fe_0    conda-forge
jupyterlab                4.2.5              pyhd8ed1ab_0    conda-forge
jupyterlab-myst           2.4.2              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.27.3             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.9              pyhd8ed1ab_0    conda-forge
kealib                    1.5.3                hee9dde6_1    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5           py311h9547e67_1    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
lcms2                     2.16                 hb7c19ff_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
leafmap                   0.36.10            pyhd8ed1ab_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libabseil                 20240116.2      cxx17_he02047a_1    conda-forge
libaec                    1.1.3                h59595ed_0    conda-forge
libarchive                3.7.4                hfca40fe_0    conda-forge
libarrow                  17.0.0          h974b3c2_11_cpu    conda-forge
libarrow-acero            17.0.0          h5888daf_11_cpu    conda-forge
libarrow-dataset          17.0.0          h5888daf_11_cpu    conda-forge
libarrow-substrait        17.0.0          hf54134d_11_cpu    conda-forge
libblas                   3.9.0           23_linux64_openblas    conda-forge
libbrotlicommon           1.1.0                hd590300_1    conda-forge
libbrotlidec              1.1.0                hd590300_1    conda-forge
libbrotlienc              1.1.0                hd590300_1    conda-forge
libcblas                  3.9.0           23_linux64_openblas    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcurl                   8.9.1                hdb1bdb2_0    conda-forge
libdeflate                1.21                 h4bc722e_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libevent                  2.1.12               hf998b51_1    conda-forge
libexpat                  2.6.2                h59595ed_0    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc-ng                 13.2.0               h807b86a_2    conda-forge
libgd                     2.3.3               hd3e95f3_10    conda-forge
libgdal                   3.9.2                ha770c72_0    conda-forge
libgdal-core              3.9.2                hba09cee_0    conda-forge
libgdal-fits              3.9.2                hdd6600c_0    conda-forge
libgdal-grib              3.9.2                h5f34788_0    conda-forge
libgdal-hdf4              3.9.2                ha39a594_0    conda-forge
libgdal-hdf5              3.9.2                ha2ed5f0_0    conda-forge
libgdal-jp2openjpeg       3.9.2                h2ebfdf0_0    conda-forge
libgdal-kea               3.9.2                h2b45729_0    conda-forge
libgdal-netcdf            3.9.2                h94e7027_0    conda-forge
libgdal-pdf               3.9.2                h0fa2cb4_0    conda-forge
libgdal-pg                3.9.2                he047751_0    conda-forge
libgdal-postgisraster     3.9.2                he047751_0    conda-forge
libgdal-tiledb            3.9.2                h9d8aadb_0    conda-forge
libgdal-xls               3.9.2                h062f1c4_0    conda-forge
libgfortran-ng            13.2.0              h69a702a_13    conda-forge
libgfortran5              13.2.0              h3d2ce59_13    conda-forge
libglib                   2.80.3               h315aac3_2    conda-forge
libgomp                   13.2.0               h807b86a_2    conda-forge
libgoogle-cloud           2.28.0               h26d7fe4_0    conda-forge
libgoogle-cloud-storage   2.28.0               ha262f82_0    conda-forge
libgrpc                   1.62.2               h15f2491_0    conda-forge
libiconv                  1.17                 hd590300_2    conda-forge
libjpeg-turbo             3.0.0                hd590300_1    conda-forge
libkml                    1.3.0             hf539b9f_1021    conda-forge
liblapack                 3.9.0           23_linux64_openblas    conda-forge
libnetcdf                 4.9.2           nompi_h135f659_114    conda-forge
libnghttp2                1.58.0               h47da74e_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libopenblas               0.3.27          pthreads_hac2b453_1    conda-forge
libparquet                17.0.0          h39682fd_11_cpu    conda-forge
libpng                    1.6.43               h2797004_0    conda-forge
libpq                     16.4                 h482b261_0    conda-forge
libprotobuf               4.25.3               h08a7969_0    conda-forge
libre2-11                 2023.09.01           h5a48ba9_2    conda-forge
librsvg                   2.58.3               h9564881_0    conda-forge
librttopo                 1.1.0               hc670b87_16    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libspatialindex           2.0.0                he02047a_0    conda-forge
libspatialite             5.1.0                h15fa968_9    conda-forge
libsqlite                 3.46.0               hde9e2c9_0    conda-forge
libssh2                   1.11.0               h0841786_0    conda-forge
libstdcxx-ng              13.2.0               h7e041cc_2    conda-forge
libthrift                 0.20.0               h0e7cc3e_1    conda-forge
libtiff                   4.6.0                h46a8edc_4    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libuv                     1.48.0               hd590300_0    conda-forge
libwebp-base              1.4.0                hd590300_0    conda-forge
libxcb                    1.16                 hb9d3cd8_1    conda-forge
libxcrypt                 4.4.36               hd590300_1    conda-forge
libxml2                   2.12.7               he7c6b58_4    conda-forge
libzip                    1.10.1               h2629f0a_3    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
linkify-it-py             2.0.3              pyhd8ed1ab_0    conda-forge
lz4-c                     1.9.4                hcb278e6_0    conda-forge
lzo                       2.10              hd590300_1001    conda-forge
mako                      1.2.4              pyhd8ed1ab_0    conda-forge
mapclassify               2.8.0              pyhd8ed1ab_0    conda-forge
maplibre                  0.2.6              pyhd8ed1ab_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.3           py311h459d7ec_1    conda-forge
matplotlib-base           3.9.2           py311h74b4f7c_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.4.1              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
minizip                   4.0.7                h401b404_0    conda-forge
mistune                   3.0.1              pyhd8ed1ab_0    conda-forge
multidict                 6.0.5           py311h459d7ec_0    conda-forge
multipledispatch          0.6.0              pyhd8ed1ab_1    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
narwhals                  1.5.5              pyhd8ed1ab_0    conda-forge
nbclassic                 1.0.0              pyhb4ecaf3_1    conda-forge
nbclient                  0.8.0              pyhd8ed1ab_0    conda-forge
nbconvert                 7.16.4                   pypi_0    pypi
nbconvert-pandoc          7.9.2              pyhd8ed1ab_0    conda-forge
nbformat                  5.9.2              pyhd8ed1ab_0    conda-forge
nbgitpuller               1.2.1              pyhd8ed1ab_0    conda-forge
ncurses                   6.5                  he02047a_1    conda-forge
nest-asyncio              1.5.8              pyhd8ed1ab_0    conda-forge
networkx                  3.3                pyhd8ed1ab_1    conda-forge
nodejs                    18.20.3              hc19f0b3_1    conda-forge
notebook                  7.2.2              pyhd8ed1ab_0    conda-forge
notebook-shim             0.2.3              pyhd8ed1ab_0    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.103                h593d115_0    conda-forge
numpy                     1.26.4          py311h64a7726_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
openjpeg                  2.5.2                h488ebb8_0    conda-forge
openssl                   3.3.1                hb9d3cd8_3    conda-forge
orc                       2.0.2                h669347b_0    conda-forge
overrides                 7.7.0              pyhd8ed1ab_0    conda-forge
packaging                 23.2               pyhd8ed1ab_0    conda-forge
pamela                    1.1.0              pyh1a96a4e_0    conda-forge
pandas                    2.2.2           py311h14de704_1    conda-forge
pandoc                    3.3                  ha770c72_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
pango                     1.54.0               h4c5309f_1    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
parsy                     2.1                pyhd8ed1ab_0    conda-forge
patsy                     0.5.6              pyhd8ed1ab_0    conda-forge
pcre2                     10.44                hba22ea6_2    conda-forge
perl                      5.32.1          7_hd590300_perl5    conda-forge
pexpect                   4.8.0              pyh1a96a4e_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.4.0          py311h82a398c_0    conda-forge
pins                      0.8.6              pyhd8ed1ab_0    conda-forge
pip                       24.2               pyh8b19718_1    conda-forge
pixman                    0.43.2               h59595ed_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_1    conda-forge
platformdirs              3.11.0             pyhd8ed1ab_0    conda-forge
playwright                1.46.0                   pypi_0    pypi
plotly                    5.23.0             pyhd8ed1ab_0    conda-forge
pmtiles                   3.4.0              pyhd8ed1ab_0    conda-forge
poppler                   24.08.0              h47131b8_1    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                16.4                 ha8faf9a_0    conda-forge
proj                      9.4.1                h54d7996_1    conda-forge
prometheus_client         0.17.1             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.39             pyha770c72_0    conda-forge
prompt_toolkit            3.0.39               hd8ed1ab_0    conda-forge
pscript                   0.7.7              pyhd8ed1ab_0    conda-forge
psutil                    5.9.5           py311h459d7ec_1    conda-forge
psygnal                   0.11.1             pyhd8ed1ab_0    conda-forge
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   17.0.0          py311hbd00459_1    conda-forge
pyarrow-core              17.0.0          py311h4510849_1_cpu    conda-forge
pyarrow-hotfix            0.6                pyhd8ed1ab_0    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycrs                     1.0.2                      py_0    conda-forge
pycurl                    7.45.3          py311h3393d6f_1    conda-forge
pydantic                  2.8.2              pyhd8ed1ab_0    conda-forge
pydantic-core             2.20.1          py311hb3a8bbb_0    conda-forge
pyee                      11.1.0                   pypi_0    pypi
pygments                  2.16.1             pyhd8ed1ab_0    conda-forge
pyjwt                     2.8.0              pyhd8ed1ab_0    conda-forge
pyopenssl                 23.2.0             pyhd8ed1ab_1    conda-forge
pyparsing                 3.1.4              pyhd8ed1ab_0    conda-forge
pyproj                    3.6.1           py311ha1c4eca_8    conda-forge
pyshp                     2.3.1              pyhd8ed1ab_0    conda-forge
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pystac                    1.10.1             pyhd8ed1ab_0    conda-forge
pystac-client             0.8.3              pyhd8ed1ab_0    conda-forge
python                    3.11.9          hb806964_0_cpython    conda-forge
python-box                7.2.0           py311h331c9d8_0    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-duckdb             1.0.0           py311hf86e51f_0    conda-forge
python-fastjsonschema     2.18.1             pyhd8ed1ab_0    conda-forge
python-graphviz           0.20.3             pyh717bed2_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-multipart          0.0.9              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python-xxhash             3.5.0           py311h61187de_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2023.3.post1       pyhd8ed1ab_0    conda-forge
pyyaml                    6.0.1           py311h459d7ec_1    conda-forge
pyzmq                     24.0.1          py311ha4b6469_1    conda-forge
qhull                     2020.2               h434a139_5    conda-forge
questionary               2.0.1              pyhd8ed1ab_0    conda-forge
re2                       2023.09.01           h7f4b329_2    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.30.2             pyhd8ed1ab_0    conda-forge
regex                     2024.7.24       py311h61187de_0    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
rpds-py                   0.10.6          py311h46250e7_0    conda-forge
rtree                     1.3.0           py311h51bcefd_1    conda-forge
ruamel.yaml               0.17.35         py311h459d7ec_0    conda-forge
ruamel.yaml.clib          0.2.7           py311h459d7ec_2    conda-forge
s2n                       1.5.1                h3400bea_0    conda-forge
scikit-learn              1.5.1           py311hd632256_0    conda-forge
scipy                     1.14.0          py311h0a5b728_2    conda-forge
scooby                    0.10.0             pyhd8ed1ab_0    conda-forge
seaborn                   0.13.2               hd8ed1ab_2    conda-forge
seaborn-base              0.13.2             pyhd8ed1ab_2    conda-forge
send2trash                1.8.2              pyh41d4057_0    conda-forge
setuptools                68.2.2             pyhd8ed1ab_0    conda-forge
shapely                   2.0.6           py311h5925939_0    conda-forge
shiny                     1.0.0              pyhd8ed1ab_0    conda-forge
simpervisor               1.0.0              pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.2.1                ha2e4443_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.5                pyhd8ed1ab_1    conda-forge
spdlog                    1.14.1               hed91bc2_1    conda-forge
sqlalchemy                2.0.22          py311h459d7ec_0    conda-forge
sqlglot                   25.9.0             pyhd8ed1ab_0    conda-forge
sqlite                    3.46.0               h6d4b2fc_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
starlette                 0.38.2             pyhd8ed1ab_0    conda-forge
statsmodels               0.14.2          py311h18e1886_0    conda-forge
tenacity                  9.0.0              pyhd8ed1ab_0    conda-forge
terminado                 0.17.1             pyh41d4057_0    conda-forge
threadpoolctl             3.5.0              pyhc1e730c_0    conda-forge
tiledb                    2.25.0               hdf2e2d9_9    conda-forge
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.13          noxft_h4845f30_101    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toolz                     0.12.1             pyhd8ed1ab_0    conda-forge
tornado                   6.3.3           py311h459d7ec_1    conda-forge
tqdm                      4.66.5             pyhd8ed1ab_0    conda-forge
traitlets                 5.11.2             pyhd8ed1ab_0    conda-forge
traittypes                0.2.1              pyh9f0ad1d_2    conda-forge
types-python-dateutil     2.8.19.14          pyhd8ed1ab_0    conda-forge
typing-extensions         4.12.2               hd8ed1ab_0    conda-forge
typing_extensions         4.12.2             pyha770c72_0    conda-forge
typing_utils              0.1.0              pyhd8ed1ab_0    conda-forge
tzcode                    2024a                h3f72095_0    conda-forge
tzdata                    2023c                h71feb2d_0    conda-forge
uc-micro-py               1.0.3              pyhd8ed1ab_0    conda-forge
uri-template              1.3.0              pyhd8ed1ab_0    conda-forge
uriparser                 0.9.8                hac33072_0    conda-forge
urllib3                   2.0.6              pyhd8ed1ab_0    conda-forge
uvicorn                   0.30.6          py311h38be061_0    conda-forge
watchfiles                0.23.0          py311hb3a8bbb_0    conda-forge
wcwidth                   0.2.8              pyhd8ed1ab_0    conda-forge
webcolors                 1.13               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1              pyhd8ed1ab_2    conda-forge
websocket-client          1.8.0              pyhd8ed1ab_0    conda-forge
websockets                13.0            py311h61187de_0    conda-forge
wheel                     0.41.2             pyhd8ed1ab_0    conda-forge
whitebox                  2.3.5              pyhd8ed1ab_0    conda-forge
whiteboxgui               2.3.0              pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.9              pyhd8ed1ab_0    conda-forge
wrapt                     1.16.0          py311h459d7ec_0    conda-forge
xerces-c                  3.2.5                h666cd97_1    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.1.1                hd590300_0    conda-forge
xorg-libsm                1.2.4                h7391055_0    conda-forge
xorg-libx11               1.8.9                hb711507_1    conda-forge
xorg-libxau               1.0.11               hd590300_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxrender           0.9.11               hd590300_0    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xxhash                    0.8.2                hd590300_0    conda-forge
xyzservices               2024.6.0           pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
y-py                      0.5.9           py311hfe55011_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yarl                      1.9.4           py311h459d7ec_0    conda-forge
ypy-websocket             0.8.2              pyhd8ed1ab_0    conda-forge
zeromq                    4.3.5                h59595ed_0    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.3.1                h4ab18f5_1    conda-forge
zstd                      1.5.6                ha6fb4c9_0    conda-forge

Using this environment.yaml:

name: nature

channels:
- conda-forge

dependencies:

# Items required for basic level functionality
- python==3.11.*
- git==2.46.0
- jupyter-resource-usage=1.1.0
- jupyterhub==4.1.6
- jupyterlab==4.2.5
- jupyter_server==2.14.2
- notebook==7.2.2
- nbgitpuller==1.2.1

# vscode
- code-server==4.23.1
- jupyter-vscode-proxy==0.6

# other packages
- seaborn==0.13.2
- altair==5.4.1
- ibis-framework[pandas,duckdb]==9.3.0
- leafmap==0.36.10
- jupyterlab-myst==2.4.2

- pip==24.2
- pip:
  - nbconvert[webpdf]==7.16.4

My goal here is to start with a reasonable baseline and iterate toward a package selection that meets your needs. I've been throwing in the items mentioned throughout this ticket as well.

[cboettig]

Can you add gh-scoped-creds to the above?

this list looks solid for modules 1+2 (even without leafmap, which brings in a lot), but I probably have to test to be sure. We can now go without altair in modules 1+2. Other than myst I believe this all works on the main hub already which we'll be using next week, so I will be deferring any use of myst until module 2.

leafmap[maplibre] will be needed in module 3, but bumped to the latest version. (If we're lucky https://github.com/eodaGmbH/py-maplibregl/pull/97 will be merged by then!) Still figuring out the remaining spatial dependencies.

Module 4 we are hoping to run using our NAIRR allocation, which promises 'jupyter notebook' support but ... we'll see! It would be useful to have streamlit on the Berkeley machines for module 4 + final project. I think we'll also be using a bunch of langchain in module 4. (yeah we probably should just be using tokens even though I'm keen for students to experience running local models).

[felder]

@cboettig my goal is to get all of the required packages into the nature hub ASAP to provide enough lead time for testing and configuration. I do not want want to wait until a week before you need it to start trying to get stuff in there. Ideally, in the future I'd like to start being able to service package requests a semester in advance so that while the semester is going changes are somewhat limited to non working items (like seaborn earlier in this issue)

Also I'm very unfamiliar with the usage of any of these python modules/libraries you're using for your course.

For example, from your latest comment I gather you want:

streamlit==1.38.0 https://anaconda.org/conda-forge/streamlit langchain, can you please specify which conda-forge packages and versions you want specifically? https://anaconda.org/search?q=langchain

When I try leafmap[maplibre]:

(notebook) jovyan@jupyter-felder:~$ mamba install leafmap[maplibre]

Looking for: ['leafmap']

pkgs/main/linux-64 (check zst)                     Checked  0.1s
pkgs/main/noarch (check zst)                        Checked  0.0s
pkgs/r/linux-64 (check zst)                         Checked  0.0s
pkgs/r/noarch (check zst)                           Checked  0.0s
pkgs/main/noarch                                   715.6kB @   4.7MB/s  0.2s
pkgs/r/linux-64                                      1.6MB @  10.0MB/s  0.2s
pkgs/main/linux-64                                   6.4MB @  26.5MB/s  0.4s
pkgs/r/noarch                                        2.1MB @   5.2MB/s  0.2s
conda-forge/noarch                                  16.2MB @  21.1MB/s  1.0s
conda-forge/linux-64                                37.4MB @  31.2MB/s  2.4s

Pinned packages:
  - python 3.11.*

Transaction

  Prefix: /srv/conda/envs/notebook

  All requested packages already installed

(notebook) jovyan@jupyter-felder:~$ 

So if leafmap[maplibre] is not working, do you have examples or import statements that I could use for testing? Once we're a bit further along, I'm also going to encourage you to do some testing and maybe even submit your own PRs.

[cboettig]

@felder I hear you, definitely want these things to be pinned down in advance if possible too. I'm running on pure pip / venv setup on my end, so I don't fully know my way around conda-forge here (especially when it comes to how conda handles these 'optional' dependencies like maplibre, https://leafmap.org/notebooks/92_maplibre/ I believe having leafmap 36.10 and py-maplibregl installed should work...

[felder]

@cboettig yeah we can use pip as a fallback if conda does not do the right thing, you can see an example of that for nbconvert here: https://github.com/berkeley-dsep-infra/datahub/blob/8e647bf0053c689f8ece857ef724d6e89d7c06b6/deployments/nature/image/environment.yml#L31

Also one thing I'd like to do is make sure we version lock packages you know you'll be importing for your assignments once we verify they work, that way we won't have them upgrading and potentially breaking on us later.

[felder]

@cboettig ok, gh-scoped-creds is in. I'm not sure how to test maplibre, but this seems to do something:

I invite you to try out the new hub and we can start working on items that don't work or are missing.

This is the production version of the hub: https://nature.datahub.berkeley.edu

[cboettig]

looks good to me! I'll start running through my modules there. we'll see how this goes!

[cboettig]

ok on the new hub now but already realizing I forgot some things.

• jupyterlab-git (I think that's what gives us the git GUI extension, right?)
• jupytext?

[felder]

@cboettig keep those missing packages coming, and I'll get'm in there but it won't be until Tuesday.

[cboettig]

• earthaccess (latest / 0.10.0)

[felder]

@cboettig just checking in. Does everything look ok for nature hub? Packages look good? Notebooks working?

[cboettig]

@felder thanks much, it's been great so far! I owe you a few more modules we'll need in a month or so but I'm still testing. Should have a few more spatial packages to add soon.

Based on today I did have one question. Would it be possible to configure things so that the default behavior for git is

git config --system pull.rebase false

Git doesn't have any default strategy, with result that the Git interface will simply fail in any merge situation because the user hasn't set this. I know I can have the students run this config in the terminal (with --global), but I'd rather this just be a default. (Not sure if --system is the best choice or if it's easier to set things up to write a config to the user's home at startup.

[felder]

@cboettig Is this a setting that every course that might potentially use nature hub would want?

After running the above on my own server I see:

(notebook) jovyan@jupyter-felder:/srv/conda/etc$ git config --list --system --show-origin
file:/srv/conda/envs/notebook/etc/gitconfig     http.sslverify=true
file:/srv/conda/envs/notebook/etc/gitconfig     http.sslcapath=/srv/conda/envs/notebook/ssl/cacert.pem
file:/srv/conda/envs/notebook/etc/gitconfig     http.sslcainfo=/srv/conda/envs/notebook/ssl/cacert.pem
file:/srv/conda/envs/notebook/etc/gitconfig     pull.rebase=false

Given that the $(prefix) for --system appears to be /srv/conda/envs/notebook/etc/gitconfig I can add a postBuild step (https://github.com/berkeley-dsep-infra/nature-user-image/blob/main/postBuild) that will set this, but doing so is going to alter the behavior across the board.

Would it be preferable to set this on a per repo basis so each course can choose the behavior they want?

Also I should add that modifying user home directories on startup is not preferable and I don't think we'd want to rely on any solutions that attempted to do that.

[cboettig]

@felder right, I believe this should be the default for all users, and I recognize we'd rather not modify home dir, which are both reasons I suggested using --system. Users can rebase pulls when they want, but to quote the git documentation,

This is a potentially dangerous mode of operation. It rewrites history, which does not bode well [...]

I know re-basing to have a 'clean history' rather than merge commits became vogue at some point, but I think re-base is an advanced technique that needs to be done with intention and one does not re-write the hash tree lightly. Originally this was how git always behaved, (I think the ability to make pull.rebase true as a configuration was introduced in git 1.7.9), and most documentation/stackoverflows etc still describe pull.rebase false as the "default" strategy, though technically there really isn't a default any more. We're using the jupyterhub-git plugin to interact with git, which clearly isn't designed to handle rebase (and the dreaded 'detached HEAD' state), so I'm confident this is the safe default.

[felder]

@cboettig the git settings change will be live on nature hub prod shortly.

[balajialg]

@cboettig It seems like the changes you requested have been resolved by @felder. Can we close this issue for now? You can open a new issue when you need additional packages. Let us know!