search

nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
https://nomic.ai/gpt4all
MIT License
70.7k stars 7.71k forks source link

Issue: GPT4ALL-API docker container memory increase on each /embeddings request #1949

Closed Karan-IceApple closed 6 months ago

Karan-IceApple commented 9 months ago

Issue you'd like to raise.

I'm using GPT4ALL-API Cpu build in my laptop and modified "/embeddings" API to connect with weaviate vector database, the connection was established between them, it using "all-MiniLM-L6-v2-f16.gguf" model for embedding, weaviate uses gpt4all container to get embedding of incoming data and stores it in Database. for each "/embedding" request, GPT4all container memory was increasing 50-80 MB on each request . is there any way to flush the memory or limit the memory or where did i go wrong ?

Suggestion:

No response

cebtenzzre commented 6 months ago

gpt4all-api has been removed, see #2314.