Add support for `keep_alive` parameter for Ollama provider for generate/chat/embedding api endpoints.

JudiniLabs / code-gpt-docs

Docusaurus page

https://code-gpt-docs.vercel.app

MIT License

585 stars 64 forks source link

Add support for `keep_alive` parameter for Ollama provider for generate/chat/embedding api endpoints. #229

Open kwame-mintah opened 10 months ago

kwame-mintah commented 10 months ago

With the latest release of Ollama v0.1.23: https://github.com/ollama/ollama/releases/tag/v0.1.23

A new keep_alive parameter has been added to allow for how long the model will stay loaded into memory following the request.

Would it be possible to allow for this configuration to be passed via the extension? More than happy to contribute to this change, if pointed in the right direction.

PilarHidalgo commented 10 months ago

Hi @kwame-mintah , We are working on our last version, but I will communicate this idea to my team! Cheers, and thanks!!