search

jupyterhub / the-littlest-jupyterhub

Simple JupyterHub distribution for 1-100 users on a single server
https://tljh.jupyter.org
BSD 3-Clause "New" or "Revised" License
1.03k stars 339 forks source link

Consider where/when to use frozen environments #857

Open minrk opened 1 year ago

minrk commented 1 year ago

Proposed change

I think we should probably use frozen environments for the bootstrap step. We've learned from experience on Binder that pinning only direct dependencies is the absolute best way to ensure broken environments. Right now, tljh has a few loose pins (not exact versions), which are likely to work for much longer, but as soon as they start rejecting the latest version (i.e. as soon as the pin has any effect at all), it is likely to start breaking things.

We can use conda-lock and/or pip-compile (they are hard to combine, unfortunately). Installing from lockfile (especially for conda) can also be quite a bit quicker, because there's no dependency-solve step, and ensures a stable, reliable starting point for the bootstrap.

Alternative options

Don't do this, and make sure the pins we use are:

  1. loose (e.g. major versions only)
  2. only applied where required
  3. frequently updated, and always point to a 'currently supported' version of every pinned package

Who would use this feature?

Maintainers, because stabilising the bootstrap versions reduces pressure to keep things always updated, as updates are opt-in. Users,

(Optional): Suggest a solution

Move package specifications from bootstrap into a file. If we still need to include them in bootstrap for single-file installation, run the standard freeze step and then include the frozen spec in a generated block in bootstrap.py.

consideRatio commented 1 year ago

I think we should probably use frozen environments for the bootstrap step.

A bit confused about the environments, is my understanding correct that you consider the "hub environment" to be setup using a frozen environment, and there are the following environments?