It is not ready for azd deployment, please update code and provide instructions.

[cynthiajiangatl]

Please provide us with the following information:

This issue is for a: (mark with an x)

- [X] bug report -> please search issues before submitting
- [ ] feature request
- [X] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

deploy.ps1 had error

Any log messages given by the failure

InvalidOperation: C:\EnterpriseChatGPT\scripts\deploy.ps1:5:5 Line | 5 | $value = $line.Split('=')[1].Trim('"') | ~~~~~~~~~~ | You cannot call a method on a null-valued expression. Set-Item: C:\EnterpriseChatGPT\scripts\deploy.ps1:6:5 Line | 6 | Set-Item -Path "env:\$name" -Value $value | ~~~~~~~~~ | Cannot process argument because the value of argument "name" is null. Change the value of argument "name" to a non-null value. Environment variables set. Error: swa command line tool is not available, check pre-requisites in README.md

Expected/desired behavior

end up replacing the environment variables with something like $env:AZURE_ENV_NAME="chatgptdemo" so no need to use azd to get environment variables.

OS and Version?

Windows 11 Enterprise

Versions

22H2

Mention any other details that might be useful

client.py has errors in the code when calling search client. need to remove the additional line 134 credential=AzureKeyCredential(AZURE_SEARCH_KEY))

Since three OpenAI services are used for load balancing, need to add those to azfunctions.bicep or provide instructions on the configuration updates to be made, default configuration for the storage account was incorrect, should be the storage account for documents.

---

Thanks! We'll be in touch soon.

[lordlinus]

Thanks @cynthiajiangatl and ack on the settings. Can you do a PR to fix them please?

[ishaan-jaff]

@lordlinus @cynthiajiangatl

I'm the maintainer of LiteLLM we provide an Open source proxy for load balancing Azure + OpenAI It can process (500+ requests/second)

From this thread it looks like you're trying to load balance between OpenAI instance - I hope our solution makes it easier for you. (i'd love feedback if you're trying to do this)

Here's the quick start:

Doc: https://docs.litellm.ai/docs/simple_proxy#load-balancing---multiple-instances-of-1-model

Step 1 Create a Config.yaml

model_list:
  - model_name: gpt-4
    litellm_params:
      model: azure/chatgpt-v-2
      api_base: https://openai-gpt-4-test-v-1.openai.azure.com/
      api_version: "2023-05-15"
      api_key: 
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_key: 
      api_base: https://openai-gpt-4-test-v-2.openai.azure.com/
  - model_name: gpt-4
    litellm_params:
      model: azure/gpt-4
      api_key: 
      api_base: https://openai-gpt-4-test-v-2.openai.azure.com/

Step 2: Start the litellm proxy:

litellm --config /path/to/config.yaml

Step3 Make Request to LiteLLM proxy:

curl --location 'http://0.0.0.0:8000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
      "model": "gpt-4",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ],
    }
'