search

rackslab / Slurm-web

Open source web dashboard for Slurm HPC clusters
https://slurm-web.com
GNU General Public License v3.0
340 stars 97 forks source link

Not supported URL scheme http+unix error after upgrade Debian #350

Closed parapar closed 2 months ago

parapar commented 2 months ago

In my system:

Linux XXXXX 6.10.3-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.10.3-1 (2024-08-04) x86_64 GNU/Linux

After upgrading to the last version. I see this error in the logs and the UI says "Unable to retrieve statistics from cluster YYYY" 

sep 19 11:15:51 varda slurm-web-agent[1692868]: During handling of the above exception, another exception occurred:
sep 19 11:15:51 varda slurm-web-agent[1692868]: Traceback (most recent call last):
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/flask/app.py", line 1473, in wsgi_app
sep 19 11:15:51 varda slurm-web-agent[1692868]:     response = self.full_dispatch_request()
sep 19 11:15:51 varda slurm-web-agent[1692868]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/flask/app.py", line 882, in full_dispatch_request
sep 19 11:15:51 varda slurm-web-agent[1692868]:     rv = self.handle_user_exception(e)
sep 19 11:15:51 varda slurm-web-agent[1692868]:          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/flask/app.py", line 880, in full_dispatch_request
sep 19 11:15:51 varda slurm-web-agent[1692868]:     rv = self.dispatch_request()
sep 19 11:15:51 varda slurm-web-agent[1692868]:          ^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/flask/app.py", line 865, in dispatch_request
sep 19 11:15:51 varda slurm-web-agent[1692868]:     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/rfl/web/tokens.py", line 93, in wrapped
sep 19 11:15:51 varda slurm-web-agent[1692868]:     return view(*args, **kwargs)
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/slurmweb/views/agent.py", line 291, in stats
sep 19 11:15:51 varda slurm-web-agent[1692868]:     version = _cached_version()
sep 19 11:15:51 varda slurm-web-agent[1692868]:               ^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/slurmweb/views/agent.py", line 147, in _cached_version
sep 19 11:15:51 varda slurm-web-agent[1692868]:     return _cached_data(
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/slurmweb/views/agent.py", line 128, in _cached_data
sep 19 11:15:51 varda slurm-web-agent[1692868]:     return func(*args)
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/slurmweb/views/agent.py", line 141, in _get_version
sep 19 11:15:51 varda slurm-web-agent[1692868]:     return slurmrest(f"/slurm/v{current_app.settings.slurmrestd.version}/ping", "meta")[
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/slurmweb/views/agent.py", line 78, in slurmrest
sep 19 11:15:51 varda slurm-web-agent[1692868]:     response = session.get(f"{prefix}/{query}")
sep 19 11:15:51 varda slurm-web-agent[1692868]:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 602, in get
sep 19 11:15:51 varda slurm-web-agent[1692868]:     return self.request("GET", url, **kwargs)
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 589, in request
sep 19 11:15:51 varda slurm-web-agent[1692868]:     resp = self.send(prep, **send_kwargs)
sep 19 11:15:51 varda slurm-web-agent[1692868]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/requests/sessions.py", line 703, in send
sep 19 11:15:51 varda slurm-web-agent[1692868]:     r = adapter.send(request, **kwargs)
sep 19 11:15:51 varda slurm-web-agent[1692868]:         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
sep 19 11:15:51 varda slurm-web-agent[1692868]:   File "/usr/lib/python3/dist-packages/requests/adapters.py", line 637, in send
sep 19 11:15:51 varda slurm-web-agent[1692868]:     raise InvalidURL(e, request=request)
sep 19 11:15:51 varda slurm-web-agent[1692868]: requests.exceptions.InvalidURL: Not supported URL scheme http+unix
sep 19 11:15:51 varda slurm-web-agent[1692868]: Thread-16 (process_request_thread): [INFO] 127.0.0.1 - - [19/Sep/2024 11:15:51] "GET /v3.2.0/stats HTTP/1.1" 500 -
parapar commented 2 months ago

It seems than forcing a downgrade of the requests package to the 2.31.0 solves this issue. For those experimenting the problem:

Download the .deb from here:

And install it with apt install ./python3-requests_2.31.0+dfsg-2_all.deb

rezib commented 2 months ago

Hello @parapar, thank you so much for reporting it!

This is actually a regression in requests 2.32, as reported here: https://github.com/psf/requests/issues/6707

We have to fix our custom adapter to support unix sockets, as applied request-unixsocket2: in https://gitlab.com/thelabnyc/requests-unixsocket2/-/commit/6f3b5757adf480c360b91acf5776009136c9496a