Wrong LLMs in backend - Githubissues

telekom / create-tsi

Create-tsi is a generative AI RAG toolkit which generates AI Applications with low code.

224 stars 27 forks source link

Wrong LLMs in backend #10

Open mmarti-tsch opened 2 months ago

mmarti-tsch commented 2 months ago

It seems that upon creation of a chatbot with create-tsi, we dont' always have the LLM in the backend which was selected in the creation process. Asking the LLM, it reveals that it is using chatGPT 3.5, even though we selected mixtral.

andrej-schreiner commented 2 months ago

@marcusschiesser any idea?

marcusschiesser commented 2 months ago

@mmarti-tsch what's the MODEL value in the .env file? If that's correct my guess is it's the LLM API

Phalanx-SoftDev commented 2 months ago

I experienced the same problem

Even though I selected mistral as llm it shows me GPT-3

mohdamir commented 2 months ago

While it tells its name as GPT3.5, its still Mixtral because in .env file Mixtral is selected and LLM Hub is using that model only. We need to check this hallucination. When i have asked about the creator of the model, following was the response.

andrej-schreiner commented 2 months ago

@tattrongvu is this expected behaviour or do we need to tweak the system prompts?