search

run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
36.76k stars 5.27k forks source link

[Bug]: AzureOpenAI doesn't pass the right key to base class #8918

Closed cipri-tom closed 1 year ago

cipri-tom commented 1 year ago

Bug Description

The interface of AzureOpenAI class has changed last week, replacing the constructor arguments: OPENAI_API_BASE -> AZURE_OPENAI_ENDPOINT and OPENAI_API_KEY -> AZURE_OPENAI_API_KEY.

When processing these new arguments, they are incorrectly passed to base class OpenAI:

https://github.com/run-llama/llama_index/commit/60ac17c677f51a5a8c630dd87c01ecc15cfe26e5#diff-f578cf6f0a325ac8e87a0335a7a34a5f68e9523091cb86a7c9240d78a57b1061R98-R99

As a result, the base class is incorrectly initialized, having an empty api_base. And therefore all calls fail with APIConnectionError: Connection error., more specifically UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol. .

See below for traceback

Version

0.8.68

Steps to Reproduce


from llama_index.llms import AzureOpenAI

llm = AzureOpenAI(
            engine=os.environ["ENGINE"],
            api_key=os.environ["OPENAI_API_KEY"],
            azure_endpoint=os.environ.get("OPENAI_API_BASE"),
            api_type=os.environ.get("OPENAI_API_TYPE"),
            api_version=os.environ.get("OPENAI_API_VERSION"),
            temperature=0.0
        )
llm.chat(['doesnt matter'])

Relevant Logs/Tracbacks

{
    "name": "APIConnectionError",
    "message": "Connection error.",
    "stack": "---------------------------------------------------------------------------
UnsupportedProtocol                       Traceback (most recent call last)
File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:66, in map_httpcore_exceptions()
     65 try:
---> 66     yield
     67 except Exception as exc:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:228, in HTTPTransport.handle_request(self, request)
    227 with map_httpcore_exceptions():
--> 228     resp = self._pool.handle_request(req)
    230 assert isinstance(resp.stream, typing.Iterable)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py:215, in ConnectionPool.handle_request(self, request)
    214 if scheme == \"\":
--> 215     raise UnsupportedProtocol(
    216         \"Request URL is missing an 'http://' or 'https://' protocol.\"
    217     )
    218 if scheme not in (\"http\", \"https\", \"ws\", \"wss\"):

UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol.

The above exception was the direct cause of the following exception:

UnsupportedProtocol                       Traceback (most recent call last)
File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:858, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    857 try:
--> 858     response = self._client.send(request, auth=self.custom_auth, stream=stream)
    859     log.debug(
    860         'HTTP Request: %s %s \"%i %s\"', request.method, request.url, response.status_code, response.reason_phrase
    861     )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:901, in Client.send(self, request, stream, auth, follow_redirects)
    899 auth = self._build_request_auth(request, auth)
--> 901 response = self._send_handling_auth(
    902     request,
    903     auth=auth,
    904     follow_redirects=follow_redirects,
    905     history=[],
    906 )
    907 try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:929, in Client._send_handling_auth(self, request, auth, follow_redirects, history)
    928 while True:
--> 929     response = self._send_handling_redirects(
    930         request,
    931         follow_redirects=follow_redirects,
    932         history=history,
    933     )
    934     try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:966, in Client._send_handling_redirects(self, request, follow_redirects, history)
    964     hook(request)
--> 966 response = self._send_single_request(request)
    967 try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:1002, in Client._send_single_request(self, request)
   1001 with request_context(request=request):
-> 1002     response = transport.handle_request(request)
   1004 assert isinstance(response.stream, SyncByteStream)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:227, in HTTPTransport.handle_request(self, request)
    215 req = httpcore.Request(
    216     method=request.method,
    217     url=httpcore.URL(
   (...)
    225     extensions=request.extensions,
    226 )
--> 227 with map_httpcore_exceptions():
    228     resp = self._pool.handle_request(req)

File ~/mambaforge/envs/llm-agent/lib/python3.11/contextlib.py:155, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    154 try:
--> 155     self.gen.throw(typ, value, traceback)
    156 except StopIteration as exc:
    157     # Suppress StopIteration *unless* it's the same exception that
    158     # was passed to throw().  This prevents a StopIteration
    159     # raised inside the \"with\" statement from being suppressed.

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:83, in map_httpcore_exceptions()
     82 message = str(exc)
---> 83 raise mapped_exc(message) from exc

UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol.

During handling of the above exception, another exception occurred:

UnsupportedProtocol                       Traceback (most recent call last)
File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:66, in map_httpcore_exceptions()
     65 try:
---> 66     yield
     67 except Exception as exc:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:228, in HTTPTransport.handle_request(self, request)
    227 with map_httpcore_exceptions():
--> 228     resp = self._pool.handle_request(req)
    230 assert isinstance(resp.stream, typing.Iterable)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py:215, in ConnectionPool.handle_request(self, request)
    214 if scheme == \"\":
--> 215     raise UnsupportedProtocol(
    216         \"Request URL is missing an 'http://' or 'https://' protocol.\"
    217     )
    218 if scheme not in (\"http\", \"https\", \"ws\", \"wss\"):

UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol.

The above exception was the direct cause of the following exception:

UnsupportedProtocol                       Traceback (most recent call last)
File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:858, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    857 try:
--> 858     response = self._client.send(request, auth=self.custom_auth, stream=stream)
    859     log.debug(
    860         'HTTP Request: %s %s \"%i %s\"', request.method, request.url, response.status_code, response.reason_phrase
    861     )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:901, in Client.send(self, request, stream, auth, follow_redirects)
    899 auth = self._build_request_auth(request, auth)
--> 901 response = self._send_handling_auth(
    902     request,
    903     auth=auth,
    904     follow_redirects=follow_redirects,
    905     history=[],
    906 )
    907 try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:929, in Client._send_handling_auth(self, request, auth, follow_redirects, history)
    928 while True:
--> 929     response = self._send_handling_redirects(
    930         request,
    931         follow_redirects=follow_redirects,
    932         history=history,
    933     )
    934     try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:966, in Client._send_handling_redirects(self, request, follow_redirects, history)
    964     hook(request)
--> 966 response = self._send_single_request(request)
    967 try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:1002, in Client._send_single_request(self, request)
   1001 with request_context(request=request):
-> 1002     response = transport.handle_request(request)
   1004 assert isinstance(response.stream, SyncByteStream)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:227, in HTTPTransport.handle_request(self, request)
    215 req = httpcore.Request(
    216     method=request.method,
    217     url=httpcore.URL(
   (...)
    225     extensions=request.extensions,
    226 )
--> 227 with map_httpcore_exceptions():
    228     resp = self._pool.handle_request(req)

File ~/mambaforge/envs/llm-agent/lib/python3.11/contextlib.py:155, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    154 try:
--> 155     self.gen.throw(typ, value, traceback)
    156 except StopIteration as exc:
    157     # Suppress StopIteration *unless* it's the same exception that
    158     # was passed to throw().  This prevents a StopIteration
    159     # raised inside the \"with\" statement from being suppressed.

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:83, in map_httpcore_exceptions()
     82 message = str(exc)
---> 83 raise mapped_exc(message) from exc

UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol.

During handling of the above exception, another exception occurred:

UnsupportedProtocol                       Traceback (most recent call last)
File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:66, in map_httpcore_exceptions()
     65 try:
---> 66     yield
     67 except Exception as exc:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:228, in HTTPTransport.handle_request(self, request)
    227 with map_httpcore_exceptions():
--> 228     resp = self._pool.handle_request(req)
    230 assert isinstance(resp.stream, typing.Iterable)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpcore/_sync/connection_pool.py:215, in ConnectionPool.handle_request(self, request)
    214 if scheme == \"\":
--> 215     raise UnsupportedProtocol(
    216         \"Request URL is missing an 'http://' or 'https://' protocol.\"
    217     )
    218 if scheme not in (\"http\", \"https\", \"ws\", \"wss\"):

UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol.

The above exception was the direct cause of the following exception:

UnsupportedProtocol                       Traceback (most recent call last)
File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:858, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    857 try:
--> 858     response = self._client.send(request, auth=self.custom_auth, stream=stream)
    859     log.debug(
    860         'HTTP Request: %s %s \"%i %s\"', request.method, request.url, response.status_code, response.reason_phrase
    861     )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:901, in Client.send(self, request, stream, auth, follow_redirects)
    899 auth = self._build_request_auth(request, auth)
--> 901 response = self._send_handling_auth(
    902     request,
    903     auth=auth,
    904     follow_redirects=follow_redirects,
    905     history=[],
    906 )
    907 try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:929, in Client._send_handling_auth(self, request, auth, follow_redirects, history)
    928 while True:
--> 929     response = self._send_handling_redirects(
    930         request,
    931         follow_redirects=follow_redirects,
    932         history=history,
    933     )
    934     try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:966, in Client._send_handling_redirects(self, request, follow_redirects, history)
    964     hook(request)
--> 966 response = self._send_single_request(request)
    967 try:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_client.py:1002, in Client._send_single_request(self, request)
   1001 with request_context(request=request):
-> 1002     response = transport.handle_request(request)
   1004 assert isinstance(response.stream, SyncByteStream)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:227, in HTTPTransport.handle_request(self, request)
    215 req = httpcore.Request(
    216     method=request.method,
    217     url=httpcore.URL(
   (...)
    225     extensions=request.extensions,
    226 )
--> 227 with map_httpcore_exceptions():
    228     resp = self._pool.handle_request(req)

File ~/mambaforge/envs/llm-agent/lib/python3.11/contextlib.py:155, in _GeneratorContextManager.__exit__(self, typ, value, traceback)
    154 try:
--> 155     self.gen.throw(typ, value, traceback)
    156 except StopIteration as exc:
    157     # Suppress StopIteration *unless* it's the same exception that
    158     # was passed to throw().  This prevents a StopIteration
    159     # raised inside the \"with\" statement from being suppressed.

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/httpx/_transports/default.py:83, in map_httpcore_exceptions()
     82 message = str(exc)
---> 83 raise mapped_exc(message) from exc

UnsupportedProtocol: Request URL is missing an 'http://' or 'https://' protocol.

The above exception was the direct cause of the following exception:

APIConnectionError                        Traceback (most recent call last)
/Users/a770pl/Library/CloudStorage/OneDrive-AXA/Documents/code/llm-agent/src/claim_exploration.ipynb Cell 3 line 1
----> <a href='vscode-notebook-cell:/Users/a770pl/Library/CloudStorage/OneDrive-AXA/Documents/code/llm-agent/src/claim_exploration.ipynb#W2sZmlsZQ%3D%3D?line=0'>1</a> executor.run(claim_id=41)

File ~/Library/CloudStorage/OneDrive-AXA/Documents/code/llm-agent/src/evaluation/agent_executor.py:151, in AgentExecutor.run(self, contract_id, claim_id, comments, **kwargs)
    148 self.agent.set_log_file(logs_file)
    150 ### Run the agent
--> 151 response = self.agent.chat(message = claim, hint = hint)
    152 agent_decision, agent_solution = self.parse_response(response)
    153 if kwargs.get(\"majority_vote\", False):

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/llama_index/callbacks/utils.py:39, in trace_method.<locals>.decorator.<locals>.wrapper(self, *args, **kwargs)
     37 callback_manager = cast(CallbackManager, callback_manager)
     38 with callback_manager.as_trace(trace_id):
---> 39     return func(self, *args, **kwargs)

File ~/Library/CloudStorage/OneDrive-AXA/Documents/code/llm-agent/src/agent/custom_agent.py:131, in InsuranceAgent.chat(self, message, chat_history, hint)
    129 ### Initialize chat history so that the first step if forced (checking the policy)
    130 if not chat_history and self.policy_reader:
--> 131     chat_history, query = self.initialize_chat_history(message)
    132     observation = \"Observation: \" + str(self.policy_reader(query))
    133     chat_history.append(ChatMessage(content=observation, role=MessageRole.USER))

File ~/Library/CloudStorage/OneDrive-AXA/Documents/code/llm-agent/src/agent/custom_agent.py:193, in InsuranceAgent.initialize_chat_history(self, input)
    182 '''Initialize chat history, forcing the first step to be a policy check.'''
    184 llm = AzureOpenAI(
    185     engine=os.environ[\"ENGINE\"],
    186     api_key=os.environ[\"OPENAI_API_KEY\"],
   (...)
    190     temperature=0.0
    191 )
--> 193 paraphrased_input = self.paraphrase_input(input, llm)
    194 thought_step = self.adapt_thought_pattern(input,llm)
    195 agent_response = MESSAGE_FORMAT.format(thought=thought_step, action_input = \"{\\\"query\\\": \\\"\" + paraphrased_input + \"\\\"}\")

File ~/Library/CloudStorage/OneDrive-AXA/Documents/code/llm-agent/src/agent/custom_agent.py:220, in InsuranceAgent.paraphrase_input(self, input, llm)
    215 '''Paraphrase the client claim into a query for the read_policy tool.'''
    216 message_list = [
    217     ChatMessage(role=MessageRole.SYSTEM, content=PARAPHRASER_SYSTEM_PROMPT),
    218     ChatMessage(role=MessageRole.USER, content=PARAPHRASER_INPUT_PROMPT.format(examples = PARAPHRASER_EXAMPLES, question=input)),
    219 ]
--> 220 result = llm.chat(message_list)
    221 return result.message.content.split(\"<\")[0].strip()

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/llama_index/llms/base.py:187, in llm_chat_callback.<locals>.wrap.<locals>.wrapped_llm_chat(_self, messages, **kwargs)
    178 with wrapper_logic(_self) as callback_manager:
    179     event_id = callback_manager.on_event_start(
    180         CBEventType.LLM,
    181         payload={
   (...)
    185         },
    186     )
--> 187     f_return_val = f(_self, messages, **kwargs)
    189     if isinstance(f_return_val, Generator):
    190         # intercept the generator and add a callback to the end
    191         def wrapped_gen() -> ChatResponseGen:

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/llama_index/llms/openai.py:171, in OpenAI.chat(self, messages, **kwargs)
    169 else:
    170     chat_fn = completion_to_chat_decorator(self._complete)
--> 171 return chat_fn(messages, **kwargs)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/llama_index/llms/openai.py:224, in OpenAI._chat(self, messages, **kwargs)
    222 def _chat(self, messages: Sequence[ChatMessage], **kwargs: Any) -> ChatResponse:
    223     message_dicts = to_openai_message_dicts(messages)
--> 224     response = self._client.chat.completions.create(
    225         messages=message_dicts,
    226         stream=False,
    227         **self._get_model_kwargs(**kwargs),
    228     )
    229     openai_message = response.choices[0].message
    230     message = from_openai_message(openai_message)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_utils/_utils.py:299, in required_args.<locals>.inner.<locals>.wrapper(*args, **kwargs)
    297             msg = f\"Missing required argument: {quote(missing[0])}\"
    298     raise TypeError(msg)
--> 299 return func(*args, **kwargs)

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/resources/chat/completions.py:594, in Completions.create(self, messages, model, frequency_penalty, function_call, functions, logit_bias, max_tokens, n, presence_penalty, response_format, seed, stop, stream, temperature, tool_choice, tools, top_p, user, extra_headers, extra_query, extra_body, timeout)
    548 @required_args([\"messages\", \"model\"], [\"messages\", \"model\", \"stream\"])
    549 def create(
    550     self,
   (...)
    592     timeout: float | httpx.Timeout | None | NotGiven = NOT_GIVEN,
    593 ) -> ChatCompletion | Stream[ChatCompletionChunk]:
--> 594     return self._post(
    595         \"/chat/completions\",
    596         body=maybe_transform(
    597             {
    598                 \"messages\": messages,
    599                 \"model\": model,
    600                 \"frequency_penalty\": frequency_penalty,
    601                 \"function_call\": function_call,
    602                 \"functions\": functions,
    603                 \"logit_bias\": logit_bias,
    604                 \"max_tokens\": max_tokens,
    605                 \"n\": n,
    606                 \"presence_penalty\": presence_penalty,
    607                 \"response_format\": response_format,
    608                 \"seed\": seed,
    609                 \"stop\": stop,
    610                 \"stream\": stream,
    611                 \"temperature\": temperature,
    612                 \"tool_choice\": tool_choice,
    613                 \"tools\": tools,
    614                 \"top_p\": top_p,
    615                 \"user\": user,
    616             },
    617             completion_create_params.CompletionCreateParams,
    618         ),
    619         options=make_request_options(
    620             extra_headers=extra_headers, extra_query=extra_query, extra_body=extra_body, timeout=timeout
    621         ),
    622         cast_to=ChatCompletion,
    623         stream=stream or False,
    624         stream_cls=Stream[ChatCompletionChunk],
    625     )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:1055, in SyncAPIClient.post(self, path, cast_to, body, options, files, stream, stream_cls)
   1041 def post(
   1042     self,
   1043     path: str,
   (...)
   1050     stream_cls: type[_StreamT] | None = None,
   1051 ) -> ResponseT | _StreamT:
   1052     opts = FinalRequestOptions.construct(
   1053         method=\"post\", url=path, json_data=body, files=to_httpx_files(files), **options
   1054     )
-> 1055     return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:834, in SyncAPIClient.request(self, cast_to, options, remaining_retries, stream, stream_cls)
    825 def request(
    826     self,
    827     cast_to: Type[ResponseT],
   (...)
    832     stream_cls: type[_StreamT] | None = None,
    833 ) -> ResponseT | _StreamT:
--> 834     return self._request(
    835         cast_to=cast_to,
    836         options=options,
    837         stream=stream,
    838         stream_cls=stream_cls,
    839         remaining_retries=remaining_retries,
    840     )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:890, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    888 except Exception as err:
    889     if retries > 0:
--> 890         return self._retry_request(
    891             options,
    892             cast_to,
    893             retries,
    894             stream=stream,
    895             stream_cls=stream_cls,
    896         )
    897     raise APIConnectionError(request=request) from err
    899 return self._process_response(
    900     cast_to=cast_to,
    901     options=options,
   (...)
    904     stream_cls=stream_cls,
    905 )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:925, in SyncAPIClient._retry_request(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)
    921 # In a synchronous context we are blocking the entire thread. Up to the library user to run the client in a
    922 # different thread if necessary.
    923 time.sleep(timeout)
--> 925 return self._request(
    926     options=options,
    927     cast_to=cast_to,
    928     remaining_retries=remaining,
    929     stream=stream,
    930     stream_cls=stream_cls,
    931 )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:890, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    888 except Exception as err:
    889     if retries > 0:
--> 890         return self._retry_request(
    891             options,
    892             cast_to,
    893             retries,
    894             stream=stream,
    895             stream_cls=stream_cls,
    896         )
    897     raise APIConnectionError(request=request) from err
    899 return self._process_response(
    900     cast_to=cast_to,
    901     options=options,
   (...)
    904     stream_cls=stream_cls,
    905 )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:925, in SyncAPIClient._retry_request(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)
    921 # In a synchronous context we are blocking the entire thread. Up to the library user to run the client in a
    922 # different thread if necessary.
    923 time.sleep(timeout)
--> 925 return self._request(
    926     options=options,
    927     cast_to=cast_to,
    928     remaining_retries=remaining,
    929     stream=stream,
    930     stream_cls=stream_cls,
    931 )

File ~/mambaforge/envs/llm-agent/lib/python3.11/site-packages/openai/_base_client.py:897, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    889     if retries > 0:
    890         return self._retry_request(
    891             options,
    892             cast_to,
   (...)
    895             stream_cls=stream_cls,
    896         )
--> 897     raise APIConnectionError(request=request) from err
    899 return self._process_response(
    900     cast_to=cast_to,
    901     options=options,
   (...)
    904     stream_cls=stream_cls,
    905 )

APIConnectionError: Connection error."
}
dosubot[bot] commented 1 year ago

🤖

Hello @cipri-tom,

I'm Dosu, an AI assistant here to provide initial support with your LlamaIndex issues. It's nice to meet you! I can help you solve bugs, answer questions, and provide guidance on contributions. While you're waiting for a human maintainer, I'm here to assist in any way I can.

I'm currently reviewing the issue you've reported about the AzureOpenAI class interface. I'll provide you with a more detailed response shortly.

logan-markewich commented 1 year ago

@cipri-tom quickly checked -- My IDE is not autocompleting api_base but it is completing azure_endpoint. Tbh I think both are incorrect, it might need to be base_url ?

image

logan-markewich commented 1 year ago

@cipri-tom if you are able to test and make a PR, that would be amazing, I don't have access to azure at the moment

logan-markewich commented 1 year ago

OH you meant the env var names changed (I think)?

We should probably fetch from both env vars to maintain backwards compatibility?

cipri-tom commented 1 year ago

@logan-markewich wow, thanks for the amazingly fast response. I have to wrap off for the day, I'll test tomorrow with the changes and try a PR.

OH you meant the env var names changed (I think)?

That as well, but it's not blocking for now, as we can pass the init arguments so that the env vars are ignored. The managing of init args are wrong.

My IDE is not autocompleting api_base but it is completing azure_endpoint

That's correct. The problem is lower, inside AzureOpenAI, here:

https://github.com/run-llama/llama_index/blob/60ac17c677f51a5a8c630dd87c01ecc15cfe26e5/llama_index/llms/azure_openai.py#L98C4-L98C4

dosubot[bot] commented 1 year ago

🤖

Thank you for bringing this issue to our attention. It seems like the Azure-specific arguments are being incorrectly passed to the base class OpenAI, which is causing the APIConnectionError.

To resolve this, you can modify the AzureOpenAI class constructor to not pass the Azure-specific arguments to the base class. Here's how you can do it:

def __init__(
        self,
        mode: str = OpenAIEmbeddingMode.TEXT_SEARCH_MODE,
        model: str = OpenAIEmbeddingModelType.TEXT_EMBED_ADA_002,
        embed_batch_size: int = DEFAULT_EMBED_BATCH_SIZE,
        additional_kwargs: Optional[Dict[str, Any]] = None,
        api_key: Optional[str] = None,
        api_version: Optional[str] = None,
        # azure specific
        azure_endpoint: Optional[str] = None,
        azure_deployment: Optional[str] = None,
        deployment_name: Optional[str] = None,
        max_retries: int = 10,
        callback_manager: Optional[CallbackManager] = None,
        **kwargs: Any,
    ):
        azure_endpoint = get_from_param_or_env(
            "azure_endpoint", azure_endpoint, "AZURE_OPENAI_ENDPOINT", ""
        )

        azure_deployment = resolve_from_aliases(
            azure_deployment,
            deployment_name,
        )

        super().__init__(
            mode=mode,
            model=model,
            embed_batch_size=embed_batch_size,
            additional_kwargs=additional_kwargs,
            api_key=api_key,
            api_version=api_version,
            max_retries=max_retries,
            callback_manager=callback_manager,
            **kwargs,
        )
        self.azure_endpoint = azure_endpoint
        self.azure_deployment = azure_deployment

In this modification, the Azure-specific arguments are not passed to the base class constructor. Instead, they are directly assigned to the AzureOpenAI instance after the base class initialization. This should prevent the APIConnectionError caused by the incorrectly initialized base class.

Please try this modification and let us know if it resolves your issue.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

cipri-tom commented 1 year ago

@logan-markewich I spent a bit of time trying to untangle the various classes and inheritance. I don't think I'm up to the task for that PR because I don't have a view on the philosophy of llama_index. I see we are re-creating OpenAI and AzureOpenAI child, both of which exist in openai package. I suppose the spirit is to provide the LLM interface on top of the default ones ?

But then we should do exactly what they do in the original classes, probably (merging azure_endpoint and api_base in a single variable):

https://github.com/openai/openai-python/blob/65e29a2efa455a06deb59e243f27796c4ca2254c/src/openai/lib/azure.py#L183-L198

Plus all the other azure_* stuff, which not passed to OpenAI class, but managed locally

aymiee commented 1 year ago

@cipri-tom - Hi, I started using llama-index last night and came upon this exact issue. Until this is resolved, do you what version prior to this one that will not have this issue? I've tried llama-index-0.8.67 and still getting the same Connection error. Thanks much in advance.

cipri-tom commented 1 year ago

@aymiee you need llama-index==0.8.47 and openai==0.28.1 . Both, otherwise they're not compatible

logan-markewich commented 1 year ago

@cipri-tom @aymiee I thiiiink the latest version of llama-index should be fine to use for azure. I know several people on discord are using it without issues. Plus just today released a few fixes for edge cases.

Maybe swing by over there to get some help with azure

aymiee commented 1 year ago

@logan-markewich This has been one of the most challenging libraries because it's evolving so fast. First the Connection error.... then re-installing for the llama-index==0.8.47 and openai==0.28.1, then having to realize that GPTVectorStoreIndex -> VectorStoreIndex.... and the myriad ways of instantiating the AzureOpenAI and AzureOpenAIEmbeddings

Thank you much for following up with me; I did update llama-index and openai finally it is working! I'm ingesting large pdf's on the fly and it worked great.

logan-markewich commented 1 year ago

@aymiee yea apologies for the rapid changes 😅 openai dev day and the new python client they released (which has been its own wild ride)

Hoping things are starting to level off a bit 🙏

mmartinortiz commented 12 months ago

For others that arrive here with the same issue. I managed to use Azure OpenAI providing twice some values:

# engine and azure_deployment have the same value
llm = AzureOpenAI(
    api_key=os.getenv("OPENAI_API_KEY", ""),
    api_version=os.getenv("OPENAI_API_VERSION", ""),
    azure_deployment=os.getenv("OPENAI_ENGINE", ""),
    azure_endpoint=os.getenv("OPENAI_API_BASE", ""),
    engine=os.getenv("OPENAI_ENGINE", ""),
    max_tokens=512,
    model="gpt-35-turbo",
    temperature=0,
)

# Here again, api_base and azure_endpoint have the same values
embed_model = AzureOpenAIEmbedding(
    api_base=os.getenv("OPENAI_API_BASE", ""),
    api_key=os.getenv("OPENAI_API_KEY", ""),
    api_version=os.getenv("OPENAI_API_VERSION", ""),
    azure_deployment=os.getenv("OPENAI_EMBEDDINGS_DEPLOYMENT", ""),
    azure_endpoint=os.getenv("OPENAI_API_BASE", ""),
)

As pointed out before, the bug is how the arguments are handled. Maybe, those redundant should be just deleted.

ashishb commented 11 months ago

@mmartinortiz I don't think azure_deployment is required. However, azure_endpoint is required.