Closed aksep closed 1 month ago
+1 I am observing this bug as well
I meet same problem.Then,I change ollama to 0.2.1. it is ok.
I think this is due to using an older model without tool calling support. Updating to llama3.1
should work. #237 fixes the doc and should be the resolution.
@anthonywu: I don't think it should fail with this specific error if no tools have been passed as argument, regardless whether the model supports them or not. There may also be reasons to want to use models that don't support tools.
Quite possible that the actual bug is in the upstream server that should treat an empty list as no tools. I would guess replacing [] with None works if the json field is then omitted altogether in the server request.
Unable to reproduce your exception. I think this problem might go away if you ollama pull llama3:latest
and git pull origin main
on this repo and maybe re-do pip install .
in your venv if it was not a install -e
install.
$ date && ollama --version && ollama list | grep llama3
Wed Jul 31 09:26:21 PDT 2024
ollama version is 0.3.0
llama3:latest 365c0bd3c000 4.7 GB 40 seconds ago
then running your snippet as is, no modifications, returns a valid response json
FWIW I ran this on macOS 14.5 M1, but IMO your problem is unlikely to be a machine/os specific issue.
Yup, it doesn't happen anymore even though I haven't really updated anything since I opened this issue. I'm still on ollama-python 0.3.0, ollama 0.2.7 (docker), and llama3 365c0bd3c000. The ollama host has been rebooted a couple of times since but it's running the same image.
Funny thing is, if I pass on a tool args, it (correctly) fails with the exception, as I would expect with this version of llama3.
Weird. Thanks for checking though!
With ollama-python 0.3.0 and the latest ollama server, I'm getting systematically an exception raised, even with the basic chat example provided (e.g.
examples/chat/main.py
).Code to reproduce
Error message
Possible fix
Changing ollama/_client.py (around line 241) to remove
or []
fixes it in my tests. I haven't checked the other occurrences of calls passing tools, but I guess they would fail too.