search

mamba-org / gator

Conda environment and package management extension from within Jupyter
Other
260 stars 31 forks source link

Failed to Retrieve Packages #127

Closed hakasapl closed 3 years ago

hakasapl commented 3 years ago

Description

When loading into the conda package manager, I get the message "an error occurred while retrieving available packages", and the package list is not populated.

Weirdly, it seems to work sometimes. I launch using JupyterHub, so occasionally my launched session will work, but will fail if I try to launch again.

Reproduce

  1. Go to Settings > Conda package manager
  2. Click on one of the environments

Expected behavior

List gets populated with conda packages.

Context

I am using JupyterHub and BatchSpawner to batch JupyterLab session using the Slurm scheduler. Not sure what to make of the logs pasted below.

Command Line Output 
[I 2021-01-24 20:37:21.582 SingleUserNotebookApp.mamba_gator handlers:373] No available packages list in cache.
[I 2021-01-24 20:37:21.583 SingleUserNotebookApp log:181] 202 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/packages?1611520639672 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 2.91ms
[I 2021-01-24 20:37:22.640 SingleUserNotebookApp build_handler:47] Build is up to date
[I 2021-01-24 20:37:22.647 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/lab/api/build?1611520639220 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 3344.01ms
[I 2021-01-24 20:37:22.650 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/lab/api/settings/@jupyterlab/docmanager-extension:plugin?1611520639748 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 8.47ms
[I 2021-01-24 20:37:24.314 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/git/settings?version=0.23.3&1611520639676 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 1670.57ms
[I 2021-01-24 20:37:24.319 SingleUserNotebookApp log:181] 202 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/tasks/1?1611520642638 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 3.79ms
[I 2021-01-24 20:37:24.326 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/metrics?1611520642642 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 9.00ms
[I 2021-01-24 20:37:24.559 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/lab/api/workspaces/lab?1611520644424 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 4.94ms
[I 2021-01-24 20:37:24.820 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/lab/api/settings?1611520644689 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 80.50ms
-[I 2021-01-24 20:37:25.199 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/environments?whitelist=0&1611520644615 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 524.83ms
\[I 2021-01-24 20:37:25.289 SingleUserNotebookApp log:181] 204 PUT /panel/jhub-dev/user/hsaplakoglu_umass_edu/lab/api/workspaces/lab?1611520645230 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 7.24ms
|[I 2021-01-24 20:37:25.401 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/api/contents/home/hsaplakoglu_umass_edu?1611520645291 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 60.62ms
[E 2021-01-24 20:37:25.448 SingleUserNotebookApp log:173] {
      "X-Forwarded-Proto": "http",
      "X-Forwarded-Port": "80",
      "Connection": "close",
      "X-Forwarded-Server": "unity.rc.umass.edu",
      "X-Forwarded-Host": "unity.rc.umass.edu",
      "X-Forwarded-For": "10.1.10.8,::ffff:127.0.0.1",
      "X-Remote-User": "hsaplakoglu@umass.edu",
      "Cookie": "",
      "Accept-Language": "en-US,en;q=0.9,tr;q=0.8,la;q=0.7",
      "Accept-Encoding": "gzip, deflate, br",
      "Referer": "https://unity.rc.umass.edu/panel/jhub-dev/user/hsaplakoglu_umass_edu/lab",
      "Sec-Fetch-Dest": "empty",
      "Sec-Fetch-Mode": "cors",
      "Sec-Fetch-Site": "same-origin",
      "Accept": "*/*",
      "Content-Type": "application/json",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36",
      "Sec-Ch-Ua-Mobile": "?0",
      "X-Xsrftoken": "2|5dee9d01|ca0d3126166b92212b18eb005a6b8ca2|1611516169",
      "Dnt": "1",
      "Authorization": "token [secret]",
      "Sec-Ch-Ua": "\"Google Chrome\";v=\"87\", \" Not;A Brand\";v=\"99\", \"Chromium\";v=\"87\"",
      "Host": "unity.rc.umass.edu"
    }
[E 2021-01-24 20:37:25.448 SingleUserNotebookApp log:181] 500 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/tasks/1?1611520645395 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 1.83ms
/[I 2021-01-24 20:37:25.499 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/api/contents/home/hsaplakoglu_umass_edu?content=1&1611520645423 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 25.29ms
-[E 2021-01-24 20:37:25.606 SingleUserNotebookApp log:173] {
      "X-Forwarded-Proto": "http",
      "X-Forwarded-Port": "80",
      "Connection": "close",
      "Content-Length": "45",
      "X-Forwarded-Server": "unity.rc.umass.edu",
      "X-Forwarded-Host": "unity.rc.umass.edu",
      "X-Forwarded-For": "10.1.10.8,::ffff:127.0.0.1",
      "X-Remote-User": "hsaplakoglu@umass.edu",
      "Cookie": "",
      "Accept-Language": "en-US,en;q=0.9,tr;q=0.8,la;q=0.7",
      "Accept-Encoding": "gzip, deflate, br",
      "Referer": "https://unity.rc.umass.edu/panel/jhub-dev/user/hsaplakoglu_umass_edu/lab",
      "Sec-Fetch-Dest": "empty",
      "Sec-Fetch-Mode": "cors",
      "Sec-Fetch-Site": "same-origin",
      "Origin": "https://unity.rc.umass.edu",
      "Accept": "*/*",
      "Content-Type": "text/plain;charset=UTF-8",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36",
      "Sec-Ch-Ua-Mobile": "?0",
      "X-Xsrftoken": "2|5dee9d01|ca0d3126166b92212b18eb005a6b8ca2|1611516169",
      "Dnt": "1",
      "Authorization": "token [secret]",
      "Sec-Ch-Ua": "\"Google Chrome\";v=\"87\", \" Not;A Brand\";v=\"99\", \"Chromium\";v=\"87\"",
      "Host": "unity.rc.umass.edu"
    }
[E 2021-01-24 20:37:25.606 SingleUserNotebookApp log:181] 500 POST /panel/jhub-dev/user/hsaplakoglu_umass_edu/git/show_top_level?1611520645508 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 43.73ms
[E 2021-01-24 20:37:25.625 SingleUserNotebookApp log:173] {
      "X-Forwarded-Proto": "http",
      "X-Forwarded-Port": "80",
      "Connection": "close",
      "Content-Length": "45",
      "X-Forwarded-Server": "unity.rc.umass.edu",
      "X-Forwarded-Host": "unity.rc.umass.edu",
      "X-Forwarded-For": "10.1.10.8,::ffff:127.0.0.1",
      "X-Remote-User": "hsaplakoglu@umass.edu",
      "Cookie": "",
      "Accept-Language": "en-US,en;q=0.9,tr;q=0.8,la;q=0.7",
      "Accept-Encoding": "gzip, deflate, br",
      "Referer": "https://unity.rc.umass.edu/panel/jhub-dev/user/hsaplakoglu_umass_edu/lab",
      "Sec-Fetch-Dest": "empty",
      "Sec-Fetch-Mode": "cors",
      "Sec-Fetch-Site": "same-origin",
      "Origin": "https://unity.rc.umass.edu",
      "Accept": "*/*",
      "Content-Type": "text/plain;charset=UTF-8",
      "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36",
      "Sec-Ch-Ua-Mobile": "?0",
      "X-Xsrftoken": "2|5dee9d01|ca0d3126166b92212b18eb005a6b8ca2|1611516169",
      "Dnt": "1",
      "Authorization": "token [secret]",
      "Sec-Ch-Ua": "\"Google Chrome\";v=\"87\", \" Not;A Brand\";v=\"99\", \"Chromium\";v=\"87\"",
      "Host": "unity.rc.umass.edu"
    }
[E 2021-01-24 20:37:25.626 SingleUserNotebookApp log:181] 500 POST /panel/jhub-dev/user/hsaplakoglu_umass_edu/git/show_top_level?1611520645513 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 52.23ms
\[I 2021-01-24 20:37:25.721 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/api/contents/home/hsaplakoglu_umass_edu?content=1&1611520645596 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 26.82ms
|[I 2021-01-24 20:37:25.820 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/channels?1611520645251 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 518.39ms
\[I 2021-01-24 20:37:26.077 SingleUserNotebookApp log:181] 204 PUT /panel/jhub-dev/user/hsaplakoglu_umass_edu/lab/api/workspaces/lab?1611520646020 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 7.29ms
/[I 2021-01-24 20:37:27.500 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/environments/base?1611520645840 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 1611.14ms
-[I 2021-01-24 20:37:27.971 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/metrics?1611520647908 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 13.60ms
|[I 2021-01-24 20:37:29.014 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/environments/base?1611520647676 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 1292.29ms
/[I 2021-01-24 20:37:29.175 SingleUserNotebookApp log:181] 200 GET /panel/jhub-dev/user/hsaplakoglu_umass_edu/conda/packages?1611520649120 (hsaplakoglu_umass_edu@::ffff:127.0.0.1) 2.13ms
Browser Output 
feature.js:7 @krassowski/jupyterlab-lsp:signature settings schema could not be found and was not loaded
de @ feature.js:7
git/show_top_level?1611522769651:1 Failed to load resource: the server responded with a status of 500 (Internal Server Error)
git/show_top_level?1611522769890:1 Failed to load resource: the server responded with a status of 500 (Internal Server Error)
services.js:426 (intermediate value) is not iterable (cannot read property undefined)
(anonymous) @ services.js:426
CondaPkgPanel.js:105 Error: An error occurred while retrieving available packages.
    at u. (services.js:429)
    at Generator.next ()
    at a (services.js:5)
(anonymous) @ CondaPkgPanel.js:105
services.js:426 (intermediate value) is not iterable (cannot read property undefined)
(anonymous) @ services.js:426
CondaPkgPanel.js:105 Error: An error occurred while retrieving available packages.
    at u. (services.js:429)
    at Generator.next ()
    at a (services.js:5)
(anonymous) @ CondaPkgPanel.js:105
services.js:426 (intermediate value) is not iterable (cannot read property undefined)
(anonymous) @ services.js:426
CondaPkgPanel.js:105 Error: An error occurred while retrieving available packages.
    at u. (services.js:429)
    at Generator.next ()
    at a (services.js:5)
(anonymous) @ CondaPkgPanel.js:105

Thanks for your help!

fcollonval commented 3 years ago

Hey @hakasapl thank you for the detailed report. Could you run jupyterlab in debug mode in order to get the output of the conda commands?

hakasapl commented 3 years ago

I'm attaching the JupyterLab log on debug mode. gator.log

fcollonval commented 3 years ago

Thanks for the log @hakasapl

So the error is on the command mamba search --json and only that one (I see that listing environments and packages is working). And the error is:

      "error": "ImportError(\"cannot import name 'id' from partially initialized module 'conda._vendor.distro' (most likely due to a circular import) (/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/conda/_vendor/distro.py)\")",
      "exception_name": "ImportError",
      "exception_type": "<class 'ImportError'>",
      "traceback": "Traceback (most recent call last):\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/conda/gateways/connection/session.py\", line 60, in __call__\n    return cls._thread_local.session\nAttributeError: '_thread._local' object has no attribute 'session'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/conda/exceptions.py\", line 1079, in __call__\n    return func(*args, **kwargs)\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/mamba/mamba.py\", line 882, in exception_converter\n    raise e\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/mamba/mamba.py\", line 876, in exception_converter\n    exit_code = _wrapped_main(*args, **kwargs)\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/mamba/mamba.py\", line 835, in _wrapped_main\n    result = do_call(args, p)\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/mamba/mamba.py\", line 714, in do_call\n    exit_code = getattr(module, func_name)(args, parser)\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/conda/cli/main_search.py\", line 73, in execute\n    matches = sorted(SubdirData.query_all(spec, channel_urls, subdirs),\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-packages/conda/core/subdir_data.py\", line 120, in query_all\n    result = tuple(concat(executor.map(subdir_query, channel_urls)))\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/concurrent/futures/_base.py\", line 600, in result_iterator\n    yield fs.pop().result()\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/concurrent/futures/_base.py\", line 433, in result\n    return self.__get_result()\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/concurrent/futures/_base.py\", line 389, in __get_result\n    raise self._exception\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/concurrent/futures/thread.py\", line 52, in run\n    result = self.fn(*self.args, **self.kwargs)\n  File \"/modules/apps/miniconda/4.8.3/envs/jupyter-2-1/lib/python3.9/site-pack

You may want to report an issue on mamba (and/or conda). But before that, you could try opening a terminal a running:

conda search --json
mamba search --json

to see if they are working. If conda works and not mamba - please report an issue on mamba repository. A solution could be to downgrade mamba or conda to a older version in the meantime.

hakasapl commented 3 years ago

Yes, I've confirmed mamba fails but conda succeeds. I've filed a bug report on mamba: https://github.com/mamba-org/mamba/issues/668

Thank you for the assistance!