search

twinnydotdev / twinny

The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but completely free and 100% private.
https://twinny.dev
MIT License
3.11k stars 165 forks source link

"Embed workspace documents" doesn't work properly #384

Open makiz1999 opened 1 week ago

makiz1999 commented 1 week ago

Describe the bug I am trying to use embeddings to create context awareness of myry so that I can ask a current directo chatbot for details within a file. I have a python file with a simple function: image

And this is chatbot response: image

Logging After embedding workspace documents: image

API Provider image

Chat or Auto Complete? Chat

Model Name codellama:7b-instruct

Desktop (please complete the following information):

rjmacarthy commented 1 week ago

Hey, thanks for the report. Please could you explain why you are using codellama:7b-instruct as the embedding model?

makiz1999 commented 1 week ago

Hey, thanks for the report. Please could you explain why you are using codellama:7b-instruct as the embedding model?

Hi, I was experimenting with different models to see if the output or quality of responses would change. From my observations, responses are pretty much similar to any model. I get a similar error when using llama3.1:latest: image

image

image

The response with llama3.1:latest is different but still doesn't make sense and reference the actual file.

Do you know by chance how to fix it? Thanks

rjmacarthy commented 1 week ago

Interesting, would have to investigate, maybe something changed with LanceDB recently...

rjmacarthy commented 1 week ago

Hey, I just released version 3.19.0. I would recommend to use https://ollama.com/library/all-minilm for the embedding model, this is the one I have had best results with in the past, let me know if you find a better one.

Many thanks!

makiz1999 commented 1 week ago

Hey, I just released version 3.19.0. I would recommend to use https://ollama.com/library/all-minilm for the embedding model, this is the one I have had best results with in the past, let me know if you find a better one.

Many thanks!

Thank you so much! I will try to test it out soon. Do you have recommendations for chat and FIM models as well?

makiz1999 commented 1 week ago

I have installed all-minilm and set it up as the embedding model. It still doesn't work as expected unfortunately and can't reference other files. Here is the error after pressing 'Embed workspace documents'. image

rjmacarthy commented 1 week ago

I think maybe your folder has a space in the name? Please try to remove the space for now. I'll fix that bug in the next release.

Many thanks.