search

FAIRmat-NFDI / nomad-distro-template

An example repository for creating a nomad distribution with custom plugins.
Apache License 2.0
7 stars 2 forks source link

`localhost` not being picked inside JupyterHub #24

Open ka-sarthak opened 1 month ago

ka-sarthak commented 1 month ago

When making API calls in JupyterHub on the oasis, the nomad.config.client.url containing localhost keyword does not work. localhost has to be replaced with the IP address for successful call.

This piece of code leads to an error append below.

from nomad.config import config
url = config.client.url
print(url)              ##  output: http://localhost/nomad-oasis/api

from nomad.client import ArchiveQuery
query = {
    "entry_id:any": ["Mg1VZFNJ-a9bkHDK3syj7MM21pFr"]
}
ArchiveQuery(query=query, required='*', url=url).download()[0]

However, replacing the URL with this resolves the issue:

url = config.client.url.replace('localhost', '172.19.0.1')

Perhaps, some config that connects the localhost with the IP address is not being picked. I also raised this issue previously here: https://gitlab.mpcdf.mpg.de/nomad-lab/nomad-FAIR/-/issues/1968

Error:

Exception in thread Thread-11:
Traceback (most recent call last):
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py", line 72](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py#line=71), in map_httpcore_exceptions
    yield
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py", line 377](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py#line=376), in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 216](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py#line=215), in handle_async_request
    raise exc from None
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 196](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection_pool.py#line=195), in handle_async_request
    response = await connection.handle_async_request(
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection.py", line 99](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection.py#line=98), in handle_async_request
    raise exc
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection.py", line 76](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection.py#line=75), in handle_async_request
    stream = await self._connect(request)
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection.py", line 122](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_async/connection.py#line=121), in _connect
    stream = await self._network_backend.connect_tcp(**kwargs)
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_backends/auto.py", line 30](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_backends/auto.py#line=29), in connect_tcp
    return await self._backend.connect_tcp(
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_backends/anyio.py", line 114](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_backends/anyio.py#line=113), in connect_tcp
    with map_exceptions(exc_map):
  File "[/opt/conda/lib/python3.10/contextlib.py", line 153](http://localhost/opt/conda/lib/python3.10/contextlib.py#line=152), in __exit__
    self.gen.throw(typ, value, traceback)
  File "[/opt/conda/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14](http://localhost/opt/conda/lib/python3.10/site-packages/httpcore/_exceptions.py#line=13), in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: All connection attempts failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "[/opt/conda/lib/python3.10/threading.py", line 1016](http://localhost/opt/conda/lib/python3.10/threading.py#line=1015), in _bootstrap_inner
    self.run()
  File "[/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py", line 47](http://localhost/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py#line=46), in run
    self.result = asyncio.run(self.func(*self.args, **self.kwargs))
  File "[/opt/conda/lib/python3.10/asyncio/runners.py", line 44](http://localhost/opt/conda/lib/python3.10/asyncio/runners.py#line=43), in run
    return loop.run_until_complete(main)
  File "[/opt/conda/lib/python3.10/asyncio/base_events.py", line 649](http://localhost/opt/conda/lib/python3.10/asyncio/base_events.py#line=648), in run_until_complete
    return future.result()
  File "[/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py", line 313](http://localhost/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py#line=312), in _fetch_async
    response = await session.post(
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_client.py", line 1905](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_client.py#line=1904), in post
    return await self.request(
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_client.py", line 1585](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_client.py#line=1584), in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_client.py", line 1674](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_client.py#line=1673), in send
    response = await self._send_handling_auth(
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_client.py", line 1702](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_client.py#line=1701), in _send_handling_auth
    response = await self._send_handling_redirects(
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_client.py", line 1739](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_client.py#line=1738), in _send_handling_redirects
    response = await self._send_single_request(request)
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_client.py", line 1776](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_client.py#line=1775), in _send_single_request
    response = await transport.handle_async_request(request)
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py", line 376](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py#line=375), in handle_async_request
    with map_httpcore_exceptions():
  File "[/opt/conda/lib/python3.10/contextlib.py", line 153](http://localhost/opt/conda/lib/python3.10/contextlib.py#line=152), in __exit__
    self.gen.throw(typ, value, traceback)
  File "[/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py", line 89](http://localhost/opt/conda/lib/python3.10/site-packages/httpx/_transports/default.py#line=88), in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: All connection attempts failed
Exception in thread Thread-13:
Traceback (most recent call last):
  File "[/opt/conda/lib/python3.10/threading.py", line 1016](http://localhost/opt/conda/lib/python3.10/threading.py#line=1015), in _bootstrap_inner
    self.run()
  File "[/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py", line 47](http://localhost/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py#line=46), in run
    self.result = asyncio.run(self.func(*self.args, **self.kwargs))
  File "[/opt/conda/lib/python3.10/asyncio/runners.py", line 44](http://localhost/opt/conda/lib/python3.10/asyncio/runners.py#line=43), in run
    return loop.run_until_complete(main)
  File "[/opt/conda/lib/python3.10/asyncio/base_events.py", line 649](http://localhost/opt/conda/lib/python3.10/asyncio/base_events.py#line=648), in run_until_complete
    return future.result()
  File "[/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py", line 392](http://localhost/opt/conda/lib/python3.10/site-packages/nomad/client/archive.py#line=391), in _download_async
    actual_number: int = min(number, len(self._entries))
TypeError: '<' not supported between instances of 'int' and 'NoneType'
ka-sarthak commented 1 month ago

After working with @blueraft, we found a fix for this issue in my locally hosted oasis. In the configs/nomad.yaml change the api_host from localhost to nomad_oasis_proxy:

services:
  api_host: 'nomad_oasis_proxy'
  api_base_path: '/nomad-oasis'

With this change, the nomad-lab inside the JupyterHub will have nomad.client.url as http://nomad_oasis_proxy/nomad-oasis/api

from nomad.config import config
url = config.client.url
print(url)              ##  output: http://nomad_oasis_proxy/nomad-oasis/api

Using ArchiveQuery with this URL works without errors.

Keep in mind, after making the above changes to the configs/nomad.yaml you need to create a new oasis image.