search

chrisammon3000 / dspy-neo4j-knowledge-graph

LLM-driven automated knowledge graph construction from text using DSPy and Neo4j.
144 stars 17 forks source link

Ollama? #1

Closed qwaszaq closed 5 months ago

qwaszaq commented 5 months ago

I’d like to keep it local as much as possible and I just wonder if it is possible to let it rock on Ollama?

chrisammon3000 commented 5 months ago

@qwaszaq Yes, absolutely but you will need to write your own integration at the moment since it is just using OpenAI.

peterbull commented 5 months ago

I've got it up and running with Ollama with a small change. In run.py change:

lm = dspy.OpenAI(
    model="gpt-4",
    max_tokens=1024,
)

To this:

lm = dspy.OllamaLocal(
    "open-hermes-2-4_0", 
    max_tokens=3000, 
    model_type="chat")

Just drop in the name of your model at open-hermes-2-4_0 and make sure that Ollama is running at http://localhost:11434/.