search

reorproject / reor

Private & local AI personal knowledge management app.
https://reorproject.org
GNU Affero General Public License v3.0
6.53k stars 372 forks source link

Issue when asking questions #37

Open Govanator12 opened 4 months ago

Govanator12 commented 4 months ago

Hi!

I'm not super familiar with using .gguf files in a local environment (or anything LLM locally for that matter) but I think I've done the setup correctly:

  1. I downloaded wizardLM-7B.Q5_K_S.guff and pointed Reor to it in the settings
  2. I added a test note
  3. I asked a question about the test note and received this: image It just keeps going if it is not stopped and happens for any asked question. What have I done wrong?
samlhuillier commented 4 months ago

I would guess this is something to do with the model rather than Reor given the output. Could you try with another gguf model like Phi-2 (one of my personal favourites): https://huggingface.co/TheBloke/phi-2-GGUF/blob/main/phi-2.Q4_K_M.gguf

Govanator12 commented 4 months ago

Working with that .gguf file (even if the answer it gave was not very good lol) Thank you!

shiloh92 commented 4 months ago

Having the same issue, even with Phi-2 model suggested. Completely ignores comments, uses default RAG and embeddings settings.