search

jupyterhub / the-littlest-jupyterhub

Simple JupyterHub distribution for 1-100 users on a single server
https://tljh.jupyter.org
BSD 3-Clause "New" or "Revised" License
1.04k stars 341 forks source link

Make unit tests against 21.10 function #714

Closed consideRatio closed 2 years ago

consideRatio commented 3 years ago

Updated issue

We have integration tests against 21.10 now and they work well, but we don't have functional unit tests. Let's close this issue if we get those up and running! The current unit test failure observed is documented in the comment below: https://github.com/jupyterhub/the-littlest-jupyterhub/issues/714#issuecomment-945154101

Original issue

I'd like to see a test against the latest Ubuntu release, such as 21.10 in this case.

I suggest this is done by first merging #710 of the three PRs adding tests for various ubuntu versions, and then opening a new PR to test against 21.10.

consideRatio commented 3 years ago

I've made a proper effort to get the integration tests and unit tests functional. But this was very tricky and I failed in the end after many attempted strategies.

  1. The runs-on property can only be a ubuntu LTS version
  2. Actions like actions/setup-python will make a cached Python available for the runs-on associated version
  3. Attempts to make do-release-upgrade in the workflow to upgrade from has failed timing out without logs emitted
  4. Attempts to use container: ubuntu:21.10 alongside runs-on has failed in integration tests where we also run docker build etc.
  5. I may have succeeded setting up a 21.10 unit test by runs-on: ubuntu-20.04 and container: ubuntu:21.10. Then the failure is a Python installation step of setup.py, where we error with something like this. 
    Successfully installed attrs-21.2.0 certifi-2021.10.8 chardet-3.0.4 charset-normalizer-2.0.7 codecov-2.1.12 coverage-6.0.2 idna-3.3 iniconfig-1.1.1 packaging-21.0 pluggy-1.0.0 py-1.10.0 pyparsing-2.4.7 pytest-6.2.5 pytest-cov-3.0.0 pytest-mock-3.6.1 requests-2.26.0 toml-0.10.2 tomli-1.2.1 urllib3-1.26.7
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
Obtaining file:///__w/the-littlest-jupyterhub/the-littlest-jupyterhub
 Preparing metadata (setup.py): started
 ERROR: Command errored out with exit status 1:
  command: /srv/venv/bin/python3 -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/__w/the-littlest-jupyterhub/the-littlest-jupyterhub/setup.py'"'"'; __file__='"'"'/__w/the-littlest-jupyterhub/the-littlest-jupyterhub/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-q63l88ul
      cwd: /__w/the-littlest-jupyterhub/the-littlest-jupyterhub/
 Complete output (11 lines):
 Traceback (most recent call last):
   File "<string>", line 1, in <module>
   File "/srv/venv/lib/python3.9/site-packages/setuptools/__init__.py", line 18, in <module>
     from setuptools.dist import Distribution
   File "/srv/venv/lib/python3.9/site-packages/setuptools/dist.py", line 38, in <module>
     from setuptools import windows_support
   File "/srv/venv/lib/python3.9/site-packages/setuptools/windows_support.py", line 2, in <module>
     import ctypes
   File "/__t/Python/3.9.7/x64/lib/python3.9/ctypes/__init__.py", line 8, in <module>
     from _ctypes import Union, Structure, Array
 ImportError: libffi.so.7: cannot open shared object file: No such file or directory
 ----------------------------------------
 Preparing metadata (setup.py): finished with status 'error'
WARNING: Discarding file:///__w/the-littlest-jupyterhub/the-littlest-jupyterhub. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

To conclude

There are two tests to make against non LTS versions, the unit tests, and the integration tests. The unit tests may actually work to make, but it may also be trouble because actions/setup-python could have provided us with a python setup on ubuntu 20.04 because of it mounting a cached installation.LTS releases only as supported

Anyhow. I'll drop my effort at this point to implement this, it is just too tricky for me to get this functional properly. My suggestion is that we stick to officially supporting LTS releases only.

consideRatio commented 3 years ago

I've tracked down the test failure listed above as a one that is fixable by releasing jupyterhub-traefik-proxy that already contains a fix. See https://github.com/jupyterhub/traefik-proxy/issues/136.

manics commented 3 years ago

Are you still aiming to add this? Is it just for testing, or do you want TLJH to officially support non LTS releases of Ubuntu?

consideRatio commented 3 years ago

I'm inclined to not support more than the latest two LTS releases, but i appreciate having tests for more as a heads up.

I'd like to make tests function, not much left now i think.

consideRatio commented 3 years ago

Okay I give up, I can't get past https://github.com/jupyterhub/the-littlest-jupyterhub/issues/714#issuecomment-945154101. I've disabled the unit test against 21.10 in #710 and made a note about this issue and the comment above where it is described.

Its worth knowing that even with all dependencies installed ahead of time, this shows up for our setup.py file. I've also tried upgrading setuptools and pip without impact.

consideRatio commented 3 years ago

We now have functional tests of everything besides unit tests for 21.10 that fail as seen in https://github.com/jupyterhub/the-littlest-jupyterhub/issues/714#issuecomment-945154101, and I can't understand clearly why.