search

mxyng / discollama

Run an AI-powered Discord bot from the comfort of your laptop.
MIT License
129 stars 22 forks source link

Discollama getting 404 back from API but other containers have no issues #10

Closed clanger81 closed 8 months ago

clanger81 commented 8 months ago

Hi, I recently began to have the issue below with my discollama container. Nothing has really changed with my configuration beyond building a new discollama image. Seems like it's having difficulty hitting the api url? I've tried a few combinations or hostnames/IPs but always get the same issue. I'll preface this by saying I have a few other containers connecting to ollama that work without issue when connecting to 'http://ollama:11434/api' Logs below:

2024-01-22 16:14:21 ERROR discord.client Ignoring exception in on_message Traceback (most recent call last): File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/ollama/_client.py", line 328, in inner r.raise_for_status() File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_models.py", line 758, in raise_for_status raise HTTPStatusError(message, request=request, response=self) httpx.HTTPStatusError: Client error '404 Not Found' for url 'http://ollama:11434/api/generate' For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/discord/client.py", line 441, in _run_event await coro(*args, **kwargs) File "/mnt/discollama.py", line 104, in on_message async for part in self.generate(content, context): File "/mnt/discollama.py", line 126, in generate async for part in await self.ollama.generate(model='llama2', prompt=content, context=context, stream=True): File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/ollama/_client.py", line 330, in inner e.response.read() File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_models.py", line 810, in read self._content = b"".join(self.iter_bytes()) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_models.py", line 828, in iter_bytes for raw_bytes in self.iter_raw(): File "/root/.cache/pypoetry/virtualenvs/discollama-_erODJfD-py3.11/lib/python3.11/site-packages/httpx/_models.py", line 879, in iter_raw raise RuntimeError("Attempted to call a sync iterator on an async stream.") RuntimeError: Attempted to call a sync iterator on an async stream.

chrismuzyn commented 8 months ago

I just ran into this issue as well. If you go to line 126 you'll see that llama2 is hardcoded as the model. I didn't have llama2 pulled on my ollama server so it 404'd. I replaced llama2 with the string of the model I wanted to use, but I think piping in the variable passed in with the OLLAMA_MODEL env variable would be desirable.

clanger81 commented 8 months ago

Thanks for pointing this out, at least there's a temporary solution for custom models. I agree it would be good if the previous default behavior of running OLLAMA_MODEL from the env was restored

mxyng commented 8 months ago

This is fixed in #11. The static llama2 was originally a placeholder I forgot about.