search

pepperoni21 / ollama-rs

A Rust library allowing to interact with the Ollama API.
MIT License
367 stars 47 forks source link

Launch the ollama server from within the Rust program? #21

Closed mandroll closed 5 months ago

mandroll commented 5 months ago

Thanks for making this extension! Do you know if it's possible to set up the ollama server from within the Rust program, such that one could deploy a standalone application that runs llama2 and makes queries to it?

Feel free to close this issue once answered. Thanks again!

pepperoni21 commented 5 months ago

Hey, the ollama server being a completely external program, I'd recommend you use Docker for that.

You can create a compose file that deploys the ollama server using the official image https://hub.docker.com/r/ollama/ollama and you can create your own image of the rust app that interacts with ollama.

mandroll commented 5 months ago

Thanks!