Sometimes model fails to answer the same question

[WhoIsDoctorWho]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Repeat this for a few times in the same / separate chat session Ask a question in the chat tab "Read the information below and answer the question "When Isaac Newton published his most famous book?" Isaac Newton published his most famous book, Principia, in 1687 while he was a mathematics professor at Trinity College, Cambridge."

Any log messages given by the failure

Expected/desired behavior

Model should be consistent with the same question

OS and Version?

Windows 10

azd version?

azd version 1.3.0

Versions

Mention any other details that might be useful

That's also reproducible with questions related to the index data. It's harder to achieve but sometimes I've encountered that model gets required context and just ignores it. I'll continue my work on that and provide you with further details.

---

Thanks! We'll be in touch soon.

[mattgotteiner]

Hi, thanks for reporting this issue

Unfortunately the chat models are not deterministic and may fail to answer the question even if the source is in the context provided. Things you can try:

1. Reduce the temperature to 0
2. We can look into the reproducible output feature and try to add it to the sample . https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/reproducible-output?tabs=pyton