Closed qwaszaq closed 5 months ago
@qwaszaq Yes, absolutely but you will need to write your own integration at the moment since it is just using OpenAI.
I've got it up and running with Ollama with a small change. In run.py
change:
lm = dspy.OpenAI(
model="gpt-4",
max_tokens=1024,
)
To this:
lm = dspy.OllamaLocal(
"open-hermes-2-4_0",
max_tokens=3000,
model_type="chat")
Just drop in the name of your model at open-hermes-2-4_0
and make sure that Ollama is running at http://localhost:11434/.
I’d like to keep it local as much as possible and I just wonder if it is possible to let it rock on Ollama?