bug: `Llama 3 8B Instruct Q4` cannot recognize that I have uploaded a file

[imtuyethan]

https://discord.com/channels/1107178041848909847/1286258581750157354

Jan version

v0.5.4

Describe the Bug

When enabling retrieval and adding a document, the Nitro model keeps insisting that there is no document to get the information from. This occurs even when a document has been successfully uploaded.

Steps to Reproduce

1. Use model: Llama 3 8B Instruct Q4 (Nitro) in Jan v0.5.4 on Windows
2. Enable retrieval feature
3. Add a document
4. Try to use the model to retrieve information from the added document
5. Observe that the model claims there is no document to get information from

Screenshots / Logs

app.log

What is your OS?

• [ ] MacOS
• [X] Windows
• [ ] Linux

[imtuyethan]

Reproducible on my end (using MacOS), our RAG feature was not implemented correctly.