Adapt to openai-python v1.0.0 Beta

Kevinello commented 1 year ago

Feature request

will you update to follow the update of https://github.com/openai/openai-python/discussions/631 ?

Motivation

Many issues in langchain point that when calling async api and using openai LLM(Chains like MapReduceDocumentsChain), it will raise 'Error Response payload is not complete' or TimeoutError

After discussion, they all point to an openai issue, which will be fix in v1.0.0(now v1.0.0 beta is avaliable)

So if you can adapt to their updates, this critical issue can be fixed

Your contribution

none

dosubot[bot] commented 1 year ago

🤖

Thank you for bringing this to our attention. We understand the importance of keeping LangChain up-to-date with the latest versions of its dependencies, including openai-python.

Based on the context provided, it seems that the v1.0.0 Beta version of openai-python has introduced changes that could potentially resolve the issues you're experiencing with async API calls and the usage of openai LLM in LangChain.

We will need to thoroughly test the new version of openai-python with LangChain to ensure compatibility and resolve any issues that may arise. This will involve updating the way we make API calls in LangChain to align with the changes introduced in the v1.0.0 Beta version of openai-python.

Here's a basic example of how the code might be modified:

try:
    values["client"] = openai.ChatCompletion.create
except AttributeError:
    raise ValueError(
        "`openai` has no `ChatCompletion.create` attribute, this is likely "
        "due to an old version of the openai package. Try upgrading it "
        "with `pip install --upgrade openai`."
    )

And for async calls, you can use the _agenerate method:

async def async_generate(llm):
    resp = await llm._agenerate(["Hello, how are you?"])
    print(resp.generations[0][0].text)

Please note that this is a basic example and additional changes might be needed based on the complete codebase of LangChain. Also, remember to handle the async nature of the API calls as per your application's requirement.

We appreciate your patience as we work on this update. We will keep you posted on our progress.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

ShantanuNair commented 1 year ago

This will likely be a big change perhaps? Since azure support is dropped, for now at least, and the underlying Requests/aiohttp packages have been swapped out with htmx. I 100% agree that this would likely solve a lot of the stability and hard to reproduce response timeout and similar bugs.

Kevinello commented 1 year ago

This will likely be a big change perhaps? Since azure support is dropped, for now at least, and the underlying Requests/aiohttp packages have been swapped out with htmx. I 100% agree that this would likely solve a lot of the stability and hard to reproduce response timeout and similar bugs.

Yes, I 100% agree that it will be a big change. But I'm still troubled by the 'acall' scenario of MapReduceDocumentsChain, in which i can reproduce the error everytime i trigger it. Do you have any way to solve this temporarily?

ShantanuNair commented 1 year ago

@Kevinello Can you share how you reproduce it? I'm curious. Also - which version of langchain are you running, and in what environment?

JoshuaConcon commented 1 year ago

Hello @Kevinello , Can you share how you reproduced this issue and what version of langchain you are running? I am trying to look into the issue as well.

I am assuming since you posted this on October 13, 2023 and assuming these errors still come up on the most updated version, I am going to assume the version you were using may be between v0.0.314 and v0.0.311 (inclusive). So I will be investigating there.

Kevinello commented 1 year ago

@Kevinello Can you share how you reproduce it? I'm curious. Also - which version of langchain are you running, and in what environment?

Hello @Kevinello , Can you share how you reproduced this issue and what version of langchain you are running? I am trying to look into the issue as well.

I am assuming since you posted this on October 13, 2023 and assuming these errors still come up on the most updated version, I am going to assume the version you were using may be between v0.0.314 and v0.0.311 (inclusive). So I will be investigating there.

Sure, I have an exact scenario that requires using SequencialChain as map_chain in MapReduceDocumentsChain. So i implement a Class like this:

class MapReduceDocumentsChain(BaseCombineDocumentsChain):
    """MapReduceDocumentsChain that is compatible for SequentialChain as map_chain"""

    map_chain: Chain
    """Chain to apply to each document individually."""
    reduce_documents_chain: BaseCombineDocumentsChain
    """Chain to use to reduce the results of applying `map_chain` to each doc.
    This typically either a ReduceDocumentChain or StuffDocumentChain."""
    map_input_key: str
    """The variable name in the map_chain to put the documents content in.
    If only one input_variable in the map_chain, this need not be provided."""
    map_output_key: str
    """The key in the result of the map_chain to use as input to the reduce_documents_chain."""

    def combine_docs(
        self,
        docs: List[Document],
        token_max: Optional[int] = None,
        callbacks: Callbacks = None,
        **kwargs: Any,
    ) -> Tuple[str, dict]:
        """Combine documents in a map reduce manner.

        Combine by mapping first chain over all documents, then reducing the results.
        This reducing can be done recursively if needed (if there are many documents).
        """
        map_results = self.map_chain.apply(
            [{self.map_input_key: d.page_content, **kwargs} for d in docs],
            callbacks=callbacks,
        )
        result_docs = [
            Document(page_content=r[self.map_output_key], metadata=docs[i].metadata)
            # This uses metadata from the docs, and the textual results from `results`
            for i, r in enumerate(map_results)
        ]
        result, extra_return_dict = self.reduce_documents_chain.combine_docs(result_docs, token_max=token_max, callbacks=callbacks, **kwargs)
        return result, extra_return_dict

    async def acombine_docs(
        self,
        docs: List[Document],
        token_max: Optional[int] = None,
        callbacks: Callbacks = None,
        **kwargs: Any,
    ) -> Tuple[str, dict]:
        """Combine documents in a map reduce manner.

        Combine by mapping first chain over all documents, then reducing the results.
        This reducing can be done recursively if needed (if there are many documents).
        """
        # NOTE - this is parallelized and so it is fast.
        tasks = [
            self.map_chain.acall(
                {self.map_input_key: d.page_content, **kwargs},
                callbacks=callbacks,
            )
            for d in docs
        ]
        map_results = await asyncio.gather(*tasks)
        result_docs = [
            Document(page_content=r[self.map_output_key], metadata=docs[i].metadata)
            # This uses metadata from the docs, and the textual results from `results`
            for i, r in enumerate(map_results)
        ]
        result, extra_return_dict = await self.reduce_documents_chain.acombine_docs(result_docs, token_max=token_max, callbacks=callbacks, **kwargs)
        return result, extra_return_dict

Then i set this mapReduceChain as a part of another SequencialChain like this:

overall_chain = SequentialChain(
    chains=[
        split_chain,
        map_reduce_chain,
    ],
    input_variables=["inputs", "question"],
    output_variables=["defect_reports"],
    callbacks=[adapter.root_callback_handler],
    verbose=True,
)

Finally when i use acall on the overall_chain, it always raise Exception below:

Kevinello commented 1 year ago

Hello @Kevinello , Can you share how you reproduced this issue and what version of langchain you are running? I am trying to look into the issue as well.

I am assuming since you posted this on October 13, 2023 and assuming these errors still come up on the most updated version, I am going to assume the version you were using may be between v0.0.314 and v0.0.311 (inclusive). So I will be investigating there.

my langchain version is 0.0.311 and openai-sdk version is 0.28.1

Kevinello commented 1 year ago

This will likely be a big change perhaps? Since azure support is dropped, for now at least, and the underlying Requests/aiohttp packages have been swapped out with htmx. I 100% agree that this would likely solve a lot of the stability and hard to reproduce response timeout and similar bugs.

sry for late reply🙃

JoshuaConcon commented 1 year ago

Thank you for the help @Kevinello , I see the issue now. I am working on a PR to update openai to the v1.0.0 Beta as you mentioned

ShantanuNair commented 11 months ago

It's out of beta now: https://github.com/openai/openai-python/discussions/742

dosubot[bot] commented 8 months ago

Hi, @Kevinello

I'm helping the LangChain team manage our backlog and am marking this issue as stale. From what I understand, you raised an issue requesting an update to adapt to the changes in openai-python v1.0.0 Beta, which was causing errors and timeouts when using async API and openai LLM. There has been a discussion among maintainers, with I providing a potential solution and ShantanuNair acknowledging the potential impact of the update. You also shared specific code and version details to help others understand the issue, and JoshuaConcon confirmed working on a PR to update openai to the v1.0.0 Beta. ShantanuNair also noted that the openai-python v1.0.0 Beta is now out of beta.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days. Thank you!

langchain-ai / langchain