search

hegelai / prompttools

Open-source tools for prompt testing and experimentation, with support for both LLMs (e.g. OpenAI, LLaMA) and vector databases (e.g. Chroma, Weaviate, LanceDB).
http://prompttools.readthedocs.io
Apache License 2.0
2.56k stars 216 forks source link

Ollama Integration #39

Open steventkrawczyk opened 11 months ago

steventkrawczyk commented 11 months ago

🚀 The feature

We want to integrate with Ollama: https://github.com/jmorganca/ollama

Goal is to build an experiment, like the LlamaCpp example: https://github.com/hegelai/prompttools/blob/main/prompttools/experiment/experiments/llama_cpp_experiment.py

Motivation, pitch

Ollama is a new way to run local models. It would be good to support this so developers can compare Ollama to LlamaCpp and other models/frameworks.

Alternatives

No response

Additional context

No response

osamanatouf2 commented 11 months ago

@steventkrawczyk Can I take care of this issue?

steventkrawczyk commented 11 months ago

Sure! We had someone else interested in it but if you can raise a PR we'll check it out and get it merged 🚀

steventkrawczyk commented 11 months ago

@osamanatouf2 For more context, we need a python API to the Ollama server to run experiments. You can look at other experiments as examples.