search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
6.12k stars 4.16k forks source link

Long deployment time #1877

Open brainlie opened 3 months ago

brainlie commented 3 months ago

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Set environment variables azd up to deploy Change some frontend code azd up again to redeploy takes anywhere from 15-45 minutes to redeploy I have been facing long deployment times and I am not sure why, the resources are already created and unchanged. The data also does not need to be ingested again, why am I facing such long deployment times? Does using local environment solve this issue? I have tried using local environment but its not the same as a live server right? My changes in the code don't reflect on the local env when I refresh the page.

Any log messages given by the failure

(✓) Done: Packaging service backend

Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time.

Subscription: -- Location: East US

(-) Skipped: Didn't find new changes. Loading azd .env file from current environment... Creating Python virtual environment "app/backend/.venv"... Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)... /Users/brainlie/Desktop/brian_LLM2/.venv/lib/python3.9/site-packages/urllib3/init.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020 warnings.warn( Updating redirect URIs for client app ID 63d26673-108b-443e-bdee-1e280e47f4d5... Application update for client app id 63d26673-108b-443e-bdee-1e280e47f4d5 complete. Loading azd .env file from current environment... Creating Python virtual environment "app/backend/.venv"... Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)... Running "prepdocs.py" /Users/brainlie/Desktop/brian_LLM2/.venv/lib/python3.9/site-packages/urllib3/init.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: https://github.com/urllib3/urllib3/issues/3020 warnings.warn( Using local files: ./data/* Ensuring search index gptkbindex exists Search index gptkbindex already exists Skipping ./data/Docker Interviews.pdf, no changes detected. Skipping ./data/Data Link Protocol part 1 (1).pdf, no changes detected. Skipping ./data/CZ3005 Module 5_Reinforcement Learning.pptx, no changes detected.

Deploying services (azd deploy)

(✓) Done: Deploying service backend

SUCCESS: Your up workflow to provision and deploy to Azure completed in 44 minutes 54 seconds.

Expected/desired behavior

Faster deployment times?

OS and Version?

macOS Monterey v12.4

azd version?

azd version 1.9.5 (commit cd2b7af9995d358aab33c782614f801ac1997dde)

cforce commented 3 months ago

Same issue here >35min

Brian Lie @.***> schrieb am Do. 1. Aug. 2024 um 09:28:

This issue is for a: (mark with an x)

  • [ ] bug report -> please search issues before submitting
  • [ ] feature request
  • [x] documentation issue or request
  • [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

Set environment variables azd up to deploy Change some frontend code azd up again to redeploy takes anywhere from 15-45 minutes to redeploy I have been facing long deployment times and I am not sure why, the resources are already created and unchanged. The data also does not need to be ingested again, why am I facing such long deployment times? Does using local environment solve this issue? I have tried using local environment but its not the same as a live server right? My changes in the code don't reflect on the local env when I refresh the page.

Any log messages given by the failure

(✓) Done: Packaging service backend

  • Package Output: /var/folders/7_/jdxg1b1s6yb_46wfq4p_g0wr0000gn/T/azure-search-openai-demo-backend-azddeploy-1722494469.zip Checking if authentication should be setup... Loading azd .env file from current environment... AZURE_USE_AUTHENTICATION is set, proceeding with authentication setup... Creating Python virtual environment "app/backend/.venv"... Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)...

    /Users/brainlie/Desktop/brian_LLM2/.venv/lib/python3.9/site-packages/urllib3/ init.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: urllib3/urllib3#3020 https://github.com/urllib3/urllib3/issues/3020 warnings.warn( Setting up authentication for tenant 6377dfe3-7b8b-4246-b974-993eafe73562 Checking if application 7a9d7d17-2d85-413e-ae97-7a021e1d67d4 exists Application already exists, not creating new one Setting up server application permissions... Checking if application 63d26673-108b-443e-bdee-1e280e47f4d5 exists Application already exists, not creating new one Setting up server known client applications... Authentication setup complete.

Provisioning Azure resources (azd provision) Provisioning Azure resources can take some time.

Subscription: -- Location: East US

(-) Skipped: Didn't find new changes. Loading azd .env file from current environment... Creating Python virtual environment "app/backend/.venv"... Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)...

/Users/brainlie/Desktop/brian_LLM2/.venv/lib/python3.9/site-packages/urllib3/ init.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: urllib3/urllib3#3020 https://github.com/urllib3/urllib3/issues/3020 warnings.warn( Updating redirect URIs for client app ID 63d26673-108b-443e-bdee-1e280e47f4d5... Application update for client app id 63d26673-108b-443e-bdee-1e280e47f4d5 complete. Loading azd .env file from current environment... Creating Python virtual environment "app/backend/.venv"... Installing dependencies from "requirements.txt" into virtual environment (in quiet mode)... Running "prepdocs.py"

/Users/brainlie/Desktop/brian_LLM2/.venv/lib/python3.9/site-packages/urllib3/ init.py:35: NotOpenSSLWarning: urllib3 v2 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with 'LibreSSL 2.8.3'. See: urllib3/urllib3#3020 https://github.com/urllib3/urllib3/issues/3020 warnings.warn( Using local files: ./data/* Ensuring search index gptkbindex exists Search index gptkbindex already exists Skipping ./data/Docker Interviews.pdf, no changes detected. Skipping ./data/Data Link Protocol part 1 (1).pdf, no changes detected. Skipping ./data/CZ3005 Module 5_Reinforcement Learning.pptx, no changes detected.

Deploying services (azd deploy)

(✓) Done: Deploying service backend

  • Endpoint: --

SUCCESS: Your up workflow to provision and deploy to Azure completed in 44 minutes 54 seconds. Expected/desired behavior

Faster deployment times?

OS and Version?

macOS Monterey v12.4

azd version?

run azd version and copy paste here.

— Reply to this email directly, view it on GitHub https://github.com/Azure-Samples/azure-search-openai-demo/issues/1877, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADAARWBNMVSQM2QE5RQOQDZPHPSFAVCNFSM6AAAAABLZ6PTOSVHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ2DCNRZHE4DIMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

bnodir commented 3 months ago

Me too

kkeaikawa commented 2 months ago

I have a same problem. Though It took about 15 min before, it takes over 30 min now.

pamelafox commented 2 months ago

A few changes have happened in last few months:

I think those are the primary causes of the slower deployment time.

bnodir commented 2 months ago

@pamelafox - Thank you very much for your explanation. Regarding the first point mentioned in your comment, is it possible that the deployment process might eventually complete without successfully tracking the deployment status? During the deployment process, I have observed the following message:

Deploying services (azd deploy)

  |     ==| Deploying service backend (Deployment with tracking status failed to start within the allotted time.Resuming deployment without tracking status.)
pamelafox commented 2 months ago

@bnodir Ah interesting, I haven't seen that, but yes, if you do see that, it means it is unable to track status. So those deployments should actually appear to finish quicker, but the app may not be truly ready when you see "Success". You can look at the App Service deployment logs to see how long the pip package building takes, to see if thats the culprit. See aka.ms/appservice-logs for details on finding those logs. I'll take a look as well on my next deploy.

Ahmetay882 commented 1 month ago

@pamelafox- Açıklamanız için çok teşekkür ederim. Yorumunuzda belirtilen ilk noktayla ilgili olarak, dağıtım sürecinin dağıtım durumunu başarıyla izlemeden tamamlanması mümkün müdür? Dağıtım süreci sırasında aşağıdaki mesajı gözlemledim:

Deploying services (azd deploy)

  |     ==| Deploying service backend (Deployment with tracking status failed to start within the allotted time.Resuming deployment without tracking status.)

It looks like this to me too. Did you solve it? Can you help me if you solved it?

bnodir commented 1 month ago

@Ahmetay882 - No, not really. Actually, I've found a workaround that might help save some time. Here's how it goes:

  1. First, stop the web app in your Azure portal page. image
  2. Start the deployment process by running the azd up command in VSCode terminal.
  3. Monitor the deployment time under the 'Last deployment' section on the web app page, it usually takes from 10 to 15 minutes.
  4. Upon noticing a [Successful on ...] status, you can terminate the deployment in terminal using Ctrl + C. image
  5. Lastly, start the web app again. This step should take about 2 to 5 minutes for your app page to be fully operational.
Ahmetay882 commented 1 month ago

Thank you for the information. Which country do you choose during distribution? Can you help me with this?

bnodir commented 1 month ago

I'm using AZURE_LOCATION="eastus2".

Ahmetay882 commented 1 month ago

Thank you very much for your reply and interest. Can I consult you when I encounter a problem?

bnodir commented 1 month ago

I'm currently allocating time to work on my RAG Hack project outside of work hours, and the schedule is quite tight as I need to submit it by the 16th of September. My apologies.

Ahmetay882 commented 1 month ago

Thank you for your interest