Azure-Samples / chat-with-your-data-solution-accelerator

A Solution Accelerator for the RAG pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences. This includes most common requirements and best practices.
https://azure.microsoft.com/products/search
MIT License
656 stars 327 forks source link

`test_raises_exception_if_bad_response_code` test fails intermittently #962

Open cecheta opened 1 month ago

cecheta commented 1 month ago

Describe the bug

The test_raises_exception_if_bad_response_code test in code/tests/utilities/helpers/test_azure_computer_vision_client.py fails intermittently. The exact cause of the error cannot be determined from the logs, as a new exception is raised when it occurs.

Expected behavior

All tests should pass consistently.


Debugging information

Screenshots

If applicable, add screenshots to help explain your problem.

image

Logs

If applicable, add logs to help the engineer debug the problem.

FAILED code/tests/utilities/helpers/test_azure_computer_vision_client.py::test_raises_exception_if_bad_response_code - assert 'Call to Azur...Vision failed' == 'Call to Azur...r says no"\n}'

  + Call to Azure Computer Vision failed
  - Call to Azure Computer Vision failed with status: 500, body: {
  -     "error": "computer says no"
  - }

Tasks

To be filled in by the engineer picking up the issue

cecheta commented 1 month ago

I have managed to reproduce this by increasing the self-defined time.sleep(0.3) in the previous test test_vectorize_image_calls_computer_vision_timeout.

It seems like the time.sleep(0.3) imposed in the previous request leaks into the subsequent tests, so the 0.25s timeout is then hit

Roopan-Microsoft commented 3 weeks ago

Hi @cecheta - I see the there are no failures recently for the above-mentioned Tests.

cecheta commented 2 weeks ago

There was a failure last Monday 3rd June, has there been a fix since then?

https://github.com/Azure-Samples/chat-with-your-data-solution-accelerator/actions/runs/9353896745