Added Ollama as an inference impl - Githubissues

meta-llama / llama-stack

Model components of the Llama Stack APIs

MIT License

4.34k stars 553 forks source link

Added Ollama as an inference impl #20

Closed hardikjshah closed 3 months ago

hardikjshah commented 3 months ago

Inline Inference

Fixed non-streaming inference sever/client/event_logger code
Added tests for inline inference to verify that things are working python -m unittest tests/test_inference.py

Ollama Inference

Added non-streaming inference using ollama python apis
Added tests to verify that ollama inference can implement the inference api spec across tools, custom tools, etc.
Streaming inference and appropriate tests
[TBD] Setup instructions for ollama and configuration
[TBD] pass right request options like temperature, etc