Add support for `keep_alive` parameter for Ollama provider for generate/chat/embedding api endpoints.

JudiniLabs / code-gpt-docs

Docusaurus page

https://code-gpt-docs.vercel.app

MIT License

553 stars 58 forks source link

Add support for `keep_alive` parameter for Ollama provider for generate/chat/embedding api endpoints. #229

Open kwame-mintah opened 5 months ago

kwame-mintah commented 5 months ago

With the latest release of Ollama v0.1.23: https://github.com/ollama/ollama/releases/tag/v0.1.23

A new keep_alive parameter has been added to allow for how long the model will stay loaded into memory following the request.

Would it be possible to allow for this configuration to be passed via the extension? More than happy to contribute to this change, if pointed in the right direction.

PilarHidalgo commented 5 months ago

Hi @kwame-mintah , We are working on our last version, but I will communicate this idea to my team! Cheers, and thanks!!