search

mxyng / discollama

Run an AI-powered Discord bot from the comfort of your laptop.
MIT License
137 stars 24 forks source link

Cannot connect to Ollama server #19

Open LeGeRyChEeSe opened 5 months ago

LeGeRyChEeSe commented 5 months ago

Hello,

I can't connect discollama to my ollama server host at 127.0.0.1:11434 address.

Here is my error message when running the bot with docker and then pinging it on discord with 'Hello' :

2024-05-26 04:30:06 discollama-1  | 2024-05-26 02:30:06 INFO     discord.client logging in using static token
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.668 # <redisgears_2> could not initialize RedisAI_InitError
2024-05-26 04:30:07 discollama-1  | 2024-05-26 02:30:07 INFO     discord.gateway Shard ID None has connected to Gateway (Session ID: XXXXXXXXXXX).
2024-05-26 04:30:05 redis-1       | 
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.668 * <redisgears_2> Failed loading RedisAI API.
2024-05-26 04:30:09 discollama-1  | 2024-05-26 02:30:09 INFO     discord.discollama Ready! Invite URL: XXXXXXXXXXXXXX0
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.668 * <redisgears_2> RedisGears v2.0.20, sha='XXXXXXXXXXXXXXXXXXX', build_type='release', built_for='Linux-ubuntu22.04.x86_64', redis_version:'7.2.4', enterprise:'false'.
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.686 * <redisgears_2> Registered backend: js.
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.693 * Module 'redisgears_2' loaded from /opt/redis-stack/lib/redisgears.so
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.694 * Server initialized
2024-05-26 04:30:05 redis-1       | 8:M 26 May 2024 02:30:05.695 * Ready to accept connections tcp
2024-05-26 04:30:32 discollama-1  | 2024-05-26 02:30:32 ERROR    discord.client Ignoring exception in on_message
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 170, in try_connect
2024-05-26 04:30:32 discollama-1  |     stream = await asynclib.connect_tcp(remote_host, remote_port, local_address)
2024-05-26 04:30:32 discollama-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2258, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     await get_running_loop().create_connection(
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1085, in create_connection
2024-05-26 04:30:32 discollama-1  |     raise exceptions[0]
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 1069, in create_connection
2024-05-26 04:30:32 discollama-1  |     sock = await self._connect_sock(
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 973, in _connect_sock
2024-05-26 04:30:32 discollama-1  |     await self.sock_connect(sock, address)
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 634, in sock_connect
2024-05-26 04:30:32 discollama-1  |     return await fut
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 674, in _sock_connect_cb
2024-05-26 04:30:32 discollama-1  |     raise OSError(err, f'Connect call failed {address}')
2024-05-26 04:30:32 discollama-1  | ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 11434)
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | The above exception was the direct cause of the following exception:
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
2024-05-26 04:30:32 discollama-1  |     yield
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 114, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     stream: anyio.abc.ByteStream = await anyio.connect_tcp(
2024-05-26 04:30:32 discollama-1  |                                    ^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/anyio/_core/_sockets.py", line 232, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     raise OSError("All connection attempts failed") from cause
2024-05-26 04:30:32 discollama-1  | OSError: All connection attempts failed
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | The above exception was the direct cause of the following exception:
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 66, in map_httpcore_exceptions
2024-05-26 04:30:32 discollama-1  |     yield
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 366, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     resp = await self._pool.handle_async_request(req)
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 268, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     raise exc
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection_pool.py", line 251, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     response = await connection.handle_async_request(request)
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     raise exc
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     stream = await self._connect(request)
2024-05-26 04:30:32 discollama-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_async/connection.py", line 124, in _connect
2024-05-26 04:30:32 discollama-1  |     stream = await self._network_backend.connect_tcp(**kwargs)
2024-05-26 04:30:32 discollama-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_backends/auto.py", line 30, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     return await self._backend.connect_tcp(
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_backends/anyio.py", line 112, in connect_tcp
2024-05-26 04:30:32 discollama-1  |     with map_exceptions(exc_map):
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
2024-05-26 04:30:32 discollama-1  |     self.gen.throw(typ, value, traceback)
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
2024-05-26 04:30:32 discollama-1  |     raise to_exc(exc) from exc
2024-05-26 04:30:32 discollama-1  | httpcore.ConnectError: All connection attempts failed
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | The above exception was the direct cause of the following exception:
2024-05-26 04:30:32 discollama-1  | 
2024-05-26 04:30:32 discollama-1  | Traceback (most recent call last):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/discord/client.py", line 441, in _run_event
2024-05-26 04:30:32 discollama-1  |     await coro(*args, **kwargs)
2024-05-26 04:30:32 discollama-1  |   File "/mnt/discollama.py", line 109, in on_message
2024-05-26 04:30:32 discollama-1  |     async for part in self.generate(content, context):
2024-05-26 04:30:32 discollama-1  |   File "/mnt/discollama.py", line 131, in generate
2024-05-26 04:30:32 discollama-1  |     async for part in await self.ollama.generate(model=self.model, prompt=content, context=context, keep_alive=-1, stream=True):
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/ollama/_client.py", line 339, in inner
2024-05-26 04:30:32 discollama-1  |     async with self._client.stream(method, url, **kwargs) as r:
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
2024-05-26 04:30:32 discollama-1  |     return await anext(self.gen)
2024-05-26 04:30:32 discollama-1  |            ^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1573, in stream
2024-05-26 04:30:32 discollama-1  |     response = await self.send(
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1617, in send
2024-05-26 04:30:32 discollama-1  |     response = await self._send_handling_auth(
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1645, in _send_handling_auth
2024-05-26 04:30:32 discollama-1  |     response = await self._send_handling_redirects(
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1682, in _send_handling_redirects
2024-05-26 04:30:32 discollama-1  |     response = await self._send_single_request(request)
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_client.py", line 1719, in _send_single_request
2024-05-26 04:30:32 discollama-1  |     response = await transport.handle_async_request(request)
2024-05-26 04:30:32 discollama-1  |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 365, in handle_async_request
2024-05-26 04:30:32 discollama-1  |     with map_httpcore_exceptions():
2024-05-26 04:30:32 discollama-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 155, in __exit__
2024-05-26 04:30:32 discollama-1  |     self.gen.throw(typ, value, traceback)
2024-05-26 04:30:32 discollama-1  |   File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_transports/default.py", line 83, in map_httpcore_exceptions
2024-05-26 04:30:32 discollama-1  |     raise mapped_exc(message) from exc
2024-05-26 04:30:32 discollama-1  | httpx.ConnectError: All connection attempts failed

And this is my ollama server running locally :

C:\Users\kilian>ollama serve
2024/05/26 04:31:21 routes.go:1008: INFO server config env="map[OLLAMA_DEBUG:false OLLAMA_LLM_LIBRARY: OLLAMA_MAX_LOADED_MODELS:1 OLLAMA_MAX_QUEUE:512 OLLAMA_MAX_VRAM:0 OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:1 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:*] OLLAMA_RUNNERS_DIR:C:\\Users\\kilian\\AppData\\Local\\Programs\\Ollama\\ollama_runners OLLAMA_TMPDIR:]"
time=2024-05-26T04:31:22.002+02:00 level=INFO source=images.go:704 msg="total blobs: 24"
time=2024-05-26T04:31:22.024+02:00 level=INFO source=images.go:711 msg="total unused blobs removed: 0"
time=2024-05-26T04:31:22.026+02:00 level=INFO source=routes.go:1054 msg="Listening on 127.0.0.1:11434 (version 0.1.38)"
time=2024-05-26T04:31:22.026+02:00 level=INFO source=payload.go:44 msg="Dynamic LLM libraries [cpu cpu_avx cpu_avx2 cuda_v11.3 rocm_v5.7]"
time=2024-05-26T04:31:22.185+02:00 level=INFO source=types.go:71 msg="inference compute" id=GPU-XXXXXXX library=cuda compute=8.6 driver=12.4 name="NVIDIA GeForce RTX 3070 Ti" total="8.0 GiB" available="6.9 GiB"

Also when this error is happening the bot still typing on discord, you should add a condition to remove the typing when error happened.

MeDott29 commented 2 months ago 
OLLAMA_HOST=http://host.docker.internal:11434

worked for me.

mxyng commented 2 months ago

if discollama is running inside a container and ollama is not, OLLAMA_HOST must be updated to address the docker host since 127.0.0.1 refers to the container. for macos and windows, this is simply setting OLLAMA_HOST to host.docker.internal which is a magic hostname set up by docker desktop. it's not quite as easy for linux. you could try setting it to the host's internal address

j3bx commented 1 month ago

Workaround for linux:

  1. Open the port of the ollama server
  2. Use 0.0.0.0 as the host when starting ollama
  3. Use this IP as host in compose.yaml: ip addr show docker0 | grep -Po 'inet \K[\d.]+'