search

explodinggradients / ragas

Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines
https://docs.ragas.io
Apache License 2.0
5.64k stars 524 forks source link

Keep getting "Your authentication token is not from a valid issuer", although I got OpenAI access #1047

Open ilaychen opened 1 week ago

ilaychen commented 1 week ago

Describe the bug I'm using AzureChatOpenAI. I was able to create a model instance, to send a prompt and to get an answer:

chat_model = AzureChatOpenAI(
    base_url="https://abc.verycoolcorp.com/etx-bot/openai/deployments" + "/gpt-35-turbo",
    api_version="2024-02-15-preview",
    http_client=httpx.Client(verify=False),
    azure_ad_token=access_token,
    temperature=0
)

response = chat_model.invoke([HumanMessage(content="Is Maldini the best footballer in history?")])

Now, when I try to use Ragas methods, such as below:

generator = TestsetGenerator.from_langchain(
    chat_model,
    critic_model,
    embeddings
)

testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25})

I'm getting an SSLCertVerificationError issue:

Your authentication token is not from a valid issuer

Ragas version: latest Python version: 3.12.1

Error trace

Exception in thread Thread-28:                                                                                                                             
Traceback (most recent call last):
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1522, in _request
    response = await self._client.send(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
    stream = await self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection.py", line 154, in _connect
    stream = await stream.start_tls(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 80, in start_tls
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 71, in start_tls
    ssl_stream = await anyio.streams.tls.TLSStream.wrap(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/anyio/streams/tls.py", line 132, in wrap
    await wrapper._call_sslobject_method(ssl_object.do_handshake)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/anyio/streams/tls.py", line 140, in _call_sslobject_method
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 917, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 95, in run
    results = self.loop.run_until_complete(self._aresults())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 684, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 83, in _aresults
    raise e
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 78, in _aresults
    r = await future
        ^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/tasks.py", line 631, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 37, in sema_coro
    return await coro
           ^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 111, in wrapped_callable_async
    return counter, await callable(*args, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/embeddings/base.py", line 26, in embed_text
    embs = await self.embed_texts([text], is_async=is_async)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/embeddings/base.py", line 36, in embed_texts
    return await aembed_documents_with_retry(texts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 142, in async_wrapped
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 58, in __call__
    else:
         ^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 110, in iter
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 78, in inner
    await self.sleep(do)
       ^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/__init__.py", line 390, in <lambda>
    else:

  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 61, in __call__
    def __iter__(self) -> t.Generator[AttemptManager, None, None]:
                             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/embeddings/base.py", line 67, in aembed_documents
    return await self.embeddings.aembed_documents(texts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/langchain_openai/embeddings/base.py", line 555, in aembed_documents
    return await self._aget_len_safe_embeddings(texts, engine=engine)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/langchain_openai/embeddings/base.py", line 475, in _aget_len_safe_embeddings
    response = await self.async_client.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/resources/embeddings.py", line 215, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1790, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1493, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1546, in _request
    return await self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1615, in _retry_request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1546, in _request
    return await self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1615, in _retry_request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1556, in _request
    raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/testset/generator.py", line 206, in generate_with_langchain_docs
    self.docstore.add_documents(
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/testset/docstore.py", line 215, in add_documents
    self.add_nodes(nodes, show_progress=show_progress)
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/testset/docstore.py", line 254, in add_nodes
    raise ExceptionInRunner()
ragas.exceptions.ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.

Expected behavior To not get this SSLCertVerificationError.

Additional context Add any other context about the problem here.

dosubot[bot] commented 1 week ago

Hey there, @ilaychen! 👋 I'm here to help you out with any bugs, questions, or contributions you have. I'm Dosu, a bot that's ready to assist you while waiting for a human maintainer.

To resolve the SSLCertVerificationError related to authentication tokens when using AzureChatOpenAI and Ragas methods, you can disable SSL verification by setting the validate_base_url parameter to False when creating the AzureChatOpenAI and AzureOpenAIEmbeddings instances.

Here is the relevant code snippet:

from langchain_openai.chat_models import AzureChatOpenAI
from langchain_openai.embeddings import AzureOpenAIEmbeddings

azure_model = AzureChatOpenAI(
    openai_api_version="2023-05-15",
    azure_endpoint=azure_configs["base_url"],
    azure_deployment=azure_configs["model_deployment"],
    model=azure_configs["model_name"],
    validate_base_url=False,  # Disable SSL verification
)

azure_embeddings = AzureOpenAIEmbeddings(
    openai_api_version="2023-05-15",
    azure_endpoint=azure_configs["base_url"],
    azure_deployment=azure_configs["embedding_deployment"],
    model=azure_configs["embedding_name"],
    validate_base_url=False,  # Disable SSL verification
)

By setting validate_base_url to False, you can bypass SSL certificate verification issues [1].

To continue talking to Dosu, mention @dosu.

ilaychen commented 1 week ago

@dosu I've updated this parameter but it didn't help

dosubot[bot] commented 1 week ago

If updating the validate_base_url parameter didn't resolve the issue, you can try explicitly disabling SSL verification in the HTTP client configuration. Here's how you can do it:

  1. Disable SSL Verification in HTTP Client: Ensure that the http_client parameter in your AzureChatOpenAI instance is configured to disable SSL verification.
import httpx
from langchain_openai.chat_models import AzureChatOpenAI

# Create an HTTP client with SSL verification disabled
http_client = httpx.Client(verify=False)

# Create an instance of AzureChatOpenAI with the custom HTTP client
chat_model = AzureChatOpenAI(
    base_url="https://abc.verycoolcorp.com/etx-bot/openai/deployments/gpt-35-turbo",
    api_version="2024-02-15-preview",
    http_client=http_client,
    azure_ad_token=access_token,
    temperature=0
)
  1. Use the Updated chat_model in Ragas Methods: Ensure that the chat_model instance with the disabled SSL verification is used in your Ragas methods.
from ragas import TestsetGenerator

# Use the chat_model with SSL verification disabled
generator = TestsetGenerator.from_langchain(
    chat_model,
    critic_model,
    embeddings
)

testset = generator.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25})

This should help bypass the SSL certificate verification issue. If the problem persists, please ensure that your access token and other configurations are correct.

To continue talking to Dosu, mention @dosu.

ilaychen commented 1 week ago

@dosu I configured AzureChatOpenAI to disable SSL verification, and I used the updated chat_model, but both of them didn't solve this issue. Any other idea?

ilaychen commented 1 week ago

@dosu Here's the updated error trace:

>>> testset = generator2.generate_with_langchain_docs(documents, test_size=10, distributions={simple: 0.5, reasoning: 0.25, multi_context: 0.25})
Exception in thread Thread-30:                                                                                                                             
Traceback (most recent call last):
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1522, in _request
    response = await self._client.send(
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1661, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1689, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1726, in _send_handling_redirects
    response = await self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_client.py", line 1763, in _send_single_request
    response = await transport.handle_async_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpx/_transports/default.py", line 373, in handle_async_request
    resp = await self._pool.handle_async_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 216, in handle_async_request
    raise exc from None
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection_pool.py", line 196, in handle_async_request
    response = await connection.handle_async_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection.py", line 99, in handle_async_request
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection.py", line 76, in handle_async_request
    stream = await self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_async/connection.py", line 154, in _connect
    stream = await stream.start_tls(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 80, in start_tls
    raise exc
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/httpcore/_backends/anyio.py", line 71, in start_tls
    ssl_stream = await anyio.streams.tls.TLSStream.wrap(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/anyio/streams/tls.py", line 132, in wrap
    await wrapper._call_sslobject_method(ssl_object.do_handshake)
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/anyio/streams/tls.py", line 140, in _call_sslobject_method
    result = func(*args)
             ^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/ssl.py", line 917, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 95, in run
    results = self.loop.run_until_complete(self._aresults())
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/base_events.py", line 684, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 83, in _aresults
    raise e
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 78, in _aresults
    r = await future
        ^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/asyncio/tasks.py", line 631, in _wait_for_one
    return f.result()  # May raise f.exception().
           ^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 37, in sema_coro
    return await coro
           ^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/executor.py", line 111, in wrapped_callable_async
    return counter, await callable(*args, **kwargs)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/embeddings/base.py", line 26, in embed_text
    embs = await self.embed_texts([text], is_async=is_async)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/embeddings/base.py", line 36, in embed_texts
    return await aembed_documents_with_retry(texts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 142, in async_wrapped
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 58, in __call__
    else:
         ^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 110, in iter
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 78, in inner
    await self.sleep(do)
       ^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/__init__.py", line 390, in <lambda>
    else:

  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/tenacity/_asyncio.py", line 61, in __call__
    def __iter__(self) -> t.Generator[AttemptManager, None, None]:
                             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/embeddings/base.py", line 67, in aembed_documents
    return await self.embeddings.aembed_documents(texts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/langchain_openai/embeddings/base.py", line 555, in aembed_documents
    return await self._aget_len_safe_embeddings(texts, engine=engine)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/langchain_openai/embeddings/base.py", line 475, in _aget_len_safe_embeddings
    response = await self.async_client.create(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/resources/embeddings.py", line 215, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1790, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1493, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1546, in _request
    return await self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1615, in _retry_request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1546, in _request
    return await self._retry_request(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1615, in _retry_request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/openai/_base_client.py", line 1556, in _request
    raise APIConnectionError(request=request) from err
openai.APIConnectionError: Connection error.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/testset/generator.py", line 206, in generate_with_langchain_docs
    self.docstore.add_documents(
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/testset/docstore.py", line 215, in add_documents
    self.add_nodes(nodes, show_progress=show_progress)
  File "/Users/ilchen/Library/Python/3.12/lib/python/site-packages/ragas/testset/docstore.py", line 254, in add_nodes
    raise ExceptionInRunner()
ragas.exceptions.ExceptionInRunner: The runner thread which was running the jobs raised an exeception. Read the traceback above to debug it. You can also pass `raise_exceptions=False` incase you want to show only a warning message instead.
dosubot[bot] commented 1 week ago

Since disabling SSL verification in both the AzureChatOpenAI and the HTTP client didn't resolve the issue, you might need to ensure that the SSL certificates are correctly set up on your system. Here are a few additional steps you can take:

  1. Install Certificates: Ensure that your system has the necessary SSL certificates installed. On macOS, you can run the following command to install the certificates:

    /Applications/Python\ 3.x/Install\ Certificates.command

  2. Update httpx Configuration: If you are using httpx, you can try setting up a custom transport with SSL verification disabled:

    import httpx
from langchain_openai.chat_models import AzureChatOpenAI

# Create a custom transport with SSL verification disabled
transport = httpx.AsyncHTTPTransport(verify=False)
http_client = httpx.AsyncClient(transport=transport)

# Create an instance of AzureChatOpenAI with the custom HTTP client
chat_model = AzureChatOpenAI(
   base_url="https://abc.verycoolcorp.com/etx-bot/openai/deployments/gpt-35-turbo",
   api_version="2024-02-15-preview",
   http_client=http_client,
   azure_ad_token=access_token,
   temperature=0
)

  3. Check Environment Variables: Ensure that there are no environment variables overriding SSL settings. For example, check if REQUESTS_CA_BUNDLE or CURL_CA_BUNDLE are set and point to the correct certificate bundle.

  4. Use certifi for SSL Certificates: You can use the certifi package to provide a certificate bundle:

    import httpx
import certifi
from langchain_openai.chat_models import AzureChatOpenAI

# Create an HTTP client with a custom certificate bundle
http_client = httpx.Client(verify=certifi.where())

# Create an instance of AzureChatOpenAI with the custom HTTP client
chat_model = AzureChatOpenAI(
   base_url="https://abc.verycoolcorp.com/etx-bot/openai/deployments/gpt-35-turbo",
   api_version="2024-02-15-preview",
   http_client=http_client,
   azure_ad_token=access_token,
   temperature=0
)

  5. Debugging SSL Issues: Enable detailed logging for httpx to get more insights into the SSL handshake process:

    import logging
import httpx

# Enable detailed logging
httpx_log = logging.getLogger("httpx")
httpx_log.setLevel(logging.DEBUG)
httpx_log.addHandler(logging.StreamHandler())

# Create an HTTP client with SSL verification disabled
http_client = httpx.Client(verify=False)

# Your existing code to create AzureChatOpenAI instance

By following these steps, you should be able to diagnose and resolve the SSL certificate verification issue. If the problem persists, it might be worth checking with your network administrator to ensure there are no network-level issues affecting SSL verification.

To continue talking to Dosu, mention @dosu.