shipley-c
commented
2 weeks ago I've definitely been able to get hallucinations even with 5 and more references. I'm not trying for hallucinations. I was using the default Mistral LLM 7B. Running the development branch.
A few users have reported they were able to cause hallucination by asking for a single line of info from a collection.
Minimal: When Concierge retrieves fewer than 5 references, maybe warn that the answer would more likely be off.
Longer term: look into additional passes/analysis if too few references are returned.