Using a local LLM with LMStudio but it requires embeddings, any embedding model or a specific one?

[Mayorc1978]

Using docker and setting the BASE_URL to my host LLM running inside LMStudio I was able to make it work, at least part of it.

But it stops to work the moment he searches for embeddings: [2024-03-15 22:08:09.682] [ERROR] Unexpected endpoint or method. (POST /v1/embeddings). Returning 200 anyway

So, does it search for a specific embedding model? Or any available models? Cause in the latter case it could work using Ollama, since Ollama has embeddings Nomic Embedding for instance is pretty fast, has better performance text-embedding-ada-002 and text-embedding-3-small performance on short and long context tasks.

Usually accessed with this:

curl http://localhost:11434/api/embeddings -d '{
  "model": "nomic-embed-text",
  "prompt": "The sky is blue because of Rayleigh scattering"
}'

[Mayorc1978]

Tested with nomic-embed-text and Ollama and works as intended.

[hirowa]

Hey @Mayorc1978 where can I find the file or configuration to use it with Ollama? Hope you have a fantastic day!

[arsaboo]

@Mayorc1978 can you please provide the changes you made to make it work with Ollama...thanks!

[zx9597446]

            # Get Ollama host and embedding model from environment variables
            # Set default value for OLLAMA_HOST
            ollama_host = os.getenv("OLLAMA_HOST", "http://host.docker.internal:11434")
            # Set default value for OLLAMA_EMBEDING_MODEL
            ollama_embedding_model = os.getenv("OLLAMA_EMBEDING_MODEL", "nomic-embed-text")

            _embeddings = OllamaEmbeddings(model=ollama_embedding_model, base_url=ollama_host)

change embeddings.py works for me

[arsaboo]

@zx9597446 that will only change the embedding model, if at all.