search

langchain-ai / langchain

๐Ÿฆœ๐Ÿ”— Build context-aware reasoning applications
https://python.langchain.com
MIT License
95.28k stars 15.46k forks source link

Issue: Getting CLOSE_WAIT and too many files open error using ChatAnthropic and StuffDocumentsChain #13509

Closed swapnil-lader closed 5 months ago

swapnil-lader commented 1 year ago

Issue you'd like to raise.

Hi, I am trying claude-2 model using the ChatAnthropic library and iterating over my data to call the model end for predictions. Since it's a chain of input, I am using StuffDocumentsChain. I am facing two issues currently.

  1. CLOSE_WAIT error after the connection is established with cloud front with anthropic, it keeps creating a new connection for each iteration of data and goes into CLOSE_WAIT state after the process is completed for that iteration after some time.
  2. When too many connections go into the close wait state the application gives too many files open error due to the file descriptor handling too many connections.

Solutions Tried

  1. Giving request_default_timeout to handle the CLOSE_WAIT error.
  2. Tried creating a function try/catch and final to close any objects within the block.

Code

import warnings
warnings.filterwarnings("ignore")
import jaconv
import numpy as np
import pandas as pd
from langchain import PromptTemplate
from fuzzysearch import find_near_matches
from langchain.vectorstores import Chroma
from langchain.chat_models import ChatAnthropic
from langchain.document_loaders import PDFPlumberLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.schema import HumanMessage, SystemMessage
from langchain.schema.output_parser import OutputParserException
from langchain.chains import LLMChain, StuffDocumentsChain
from langchain.retrievers import BM25Retriever, EnsembleRetriever
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.output_parsers import StructuredOutputParser, ResponseSchema
from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplate
import time
from json.decoder import JSONDecodeError
from loguru import logger

output_parser = StructuredOutputParser.from_response_schemas([
    ResponseSchema(
        name="answer",
        description="answer to the user's question",
        type="string"
    ),
])
format_instructions = output_parser.get_format_instructions()

def load_data_(file_path):
    return PDFPlumberLoader(file_path).load_and_split()

def retriever_(docs):
    return EnsembleRetriever(
        retrievers=[
            BM25Retriever.from_documents(docs, k=4),
            Chroma.from_documents(
                docs,
                OpenAIEmbeddings(
                    chunk_size=200,
                    max_retries=60,
                    show_progress_bar=True
                )
            ).as_retriever(search_kwargs={"k": 4})
        ],
        weights=[0.5, 0.5]
    )

def qa_chain():
    start_qa = time.time()
    qa_chain_output = StuffDocumentsChain(
        llm_chain=LLMChain(
            llm=ChatAnthropic(
                model_name="claude-2",
                temperature=0,
                top_p=1,
                max_tokens_to_sample=500000,
                default_request_timeout=30
            ),
            prompt=ChatPromptTemplate(
                messages=[
                    SystemMessage(content='You are a world class & knowledgeable product catalog assistance.'),
                    HumanMessagePromptTemplate.from_template('Context:\n{context}'),
                    HumanMessagePromptTemplate.from_template('{format_instructions}'),
                    HumanMessagePromptTemplate.from_template('Questions:\n{question}'),
                    HumanMessage(content='Tips: Make sure to answer in the correct format.'),
                ],
                partial_variables={"format_instructions": format_instructions}
            )
        ),
        document_variable_name='context',
        document_prompt=PromptTemplate(
            template='Content: {page_content}',
            input_variables=['page_content'],
        )
    )
    logger.info("Finished QA Chain in {}",time.time()-start_qa)
    return qa_chain_output

def generate(input_data, retriever):
    start_generate = time.time()
    doc_query = jaconv.z2h(
        jaconv.normalize(
            "ใ‚ณใƒผใƒ‰: {}\n่ฃฝๅ“ใฎ็จฎ้กž: {}\nใƒกใƒผใ‚ซใƒผๅ“ๅ: {}".format(
                str(input_data["Product Code"]),
                str(input_data["Product Type"]),
                str(input_data["Product Name"])
            ),
            "NFKC"
        ),
        kana=False,
        digit=True,
        ascii=True
    )
    docs = RecursiveCharacterTextSplitter(
        chunk_size=512,
        chunk_overlap=128
    ).split_documents(
        retriever.get_relevant_documents(doc_query)
    )
    docs = [
        doc
        for doc in docs
        if len(
            find_near_matches(
                str(input_data["Product Code"]),
                str(doc.page_content),
                max_l_dist=1
            )
        ) > 0
    ]
    pages = list(set([str(doc.metadata["page"]) for doc in docs]))

    question = (
        "Analyze the provided context and understand to extract the following attributes as precise as possible for {}"
        "Attributes:\n"
        "{}\n"
        "Return it in this JSON format: {}. "
        "if no information found or not sure return \"None\"."
    )
    generate_output = output_parser.parse(
        qa_chain().run({
            "input_documents": docs,
            "question": question.format(
                str(input_data["Product Code"]),
                '\n'.join([
                    f" {i + 1}. What is the value of \"{attrib}\"?"
                    for i, attrib in enumerate(input_data["Attributes"])
                ]),
                str({"answer": {attrib: f"value of {attrib}" for attrib in input_data["Attributes"]}}),
            )
        })
    )["answer"], ';'.join(pages)
    logger.info("Finished QA Chain in {}",time.time()-start_generate)
    return generate_output

def predict(pdf_file_path:str, csv_file_path:str,output_csv_file_path:str):
    # load PDF data
    start_load = time.time()
    logger.info("Started loading the pdf")
    documents = load_data_(pdf_file_path)
    logger.info("Finished loading the pdf in {}",time.time()-start_load)
    try:
        start_retriever = time.time()
        logger.info("Started retriever the pdf")
        retriever = retriever_(documents)
        logger.info("Finished retriever the pdf in {}",time.time()-start_retriever)
    except Exception as e:
        logger.error(f"Error in Retriever: {e}")
    # Load CSV
    start_load_csv = time.time()
    logger.info("Started Reading Csv")
    df = pd.read_csv(
        csv_file_path,
        low_memory=False,
        dtype=object
    ).replace(np.nan, 'None')
    logger.info("Finished Reading Csv in {}",time.time()-start_load_csv)
    # Inference
    index = 0
    result_df = list()
    start_generate = time.time()
    logger.info("Started Generate Function")
    for code, dfx in df.groupby('Product Code'):
        start_generate_itr = time.time()
        try:
            logger.info("Reached index {}",index)
            logger.info("Reached index code {}",code)
            index = index + 1
            result, page = generate(
                {
                    'Product Code': code,
                    'Product Type': dfx['Product Type'].tolist()[0],
                    'Product Name': dfx['Product Name'].tolist()[0],
                    'Attributes': dfx['Attributes'].tolist()
                },
                retriever
            )
            dfx['Value'] = dfx['Attributes'].apply(lambda attrib: result.get(attrib, 'None'))
            dfx['Page'] = page
            logger.info("Generate Function 1 iterations {}",time.time()-start_generate_itr)
        except OutputParserException as e:
            dfx['Value'] = "None"
            dfx['Page'] = "None"
            logger.info("Generate Function 1 iterations {}",time.time()-start_generate_itr)
            logger.info("JSONDecodeError Exception Occurred in {}",e)
        result_df.append(dfx)
    logger.info("Finished Generate Function in {}",time.time()-start_generate)
    try:
        result_df = pd.concat(result_df)
        result_df.to_csv(output_csv_file_path)
    except FileNotFoundError as e:
        logger.error("FileNotFoundError Exception Occurred in {}",e)
    return result_df

# df = predict(f"{PDF_FILE}", f"{CSV_FILE}","output.csv")

Suggestion:

I would like to know how to handle the connection CLOSE_WAIT error since I am handling a big amount of data to be processed through Anthropic claude-2

dosubot[bot] commented 1 year ago

๐Ÿค–

Hello @swapnil-lader,

I'm an assistance bot here to help you with bugs, questions, and to guide you on how to contribute to the LangChain repository. While we're waiting for a human maintainer, I'm here to assist you as best as I can. I'm currently looking into your issue and will provide a detailed response shortly.

dosubot[bot] commented 1 year ago

๐Ÿค–

Thank you for providing a detailed description of the issue you're facing. It seems like you're encountering a common problem with handling a large number of connections, which is causing the CLOSE_WAIT error and the "too many files open" error.

The CLOSE_WAIT error typically occurs when the local end of the connection has received a FIN from the other end, but the OS is waiting for the program at the local end to actually close its connection. The issue is that the local end is not closing the connection, and it's just sitting there in the CLOSE_WAIT state.

The "too many files open" error is likely related to this, as each connection uses a file descriptor, and if connections are not being closed properly, you can hit the limit of open file descriptors.

However, to provide a more accurate solution, I would need a bit more information:

  1. Could you provide the version of LangChain you are using? You can find this by running pip show langchain in your terminal.
  2. Could you provide the version of Python you are using? You can find this by running python --version in your terminal.
  3. Could you provide the operating system you are using and its version?
  4. Could you provide the traceback of the error messages you are receiving?

This additional information will help me understand the context better and provide a more accurate solution.

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a ๐Ÿ‘ if this is helpful and ๐Ÿ‘Ž if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

swapnil-lader commented 1 year ago

๐Ÿค–

Thank you for providing a detailed description of the issue you're facing. It seems like you're encountering a common problem with handling a large number of connections, which is causing the CLOSE_WAIT error and the "too many files open" error.

The CLOSE_WAIT error typically occurs when the local end of the connection has received a FIN from the other end, but the OS is waiting for the program at the local end to actually close its connection. The issue is that the local end is not closing the connection, and it's just sitting there in the CLOSE_WAIT state.

The "too many files open" error is likely related to this, as each connection uses a file descriptor, and if connections are not being closed properly, you can hit the limit of open file descriptors.

However, to provide a more accurate solution, I would need a bit more information:

  1. Could you provide the version of LangChain you are using? You can find this by running pip show langchain in your terminal.
  2. Could you provide the version of Python you are using? You can find this by running python --version in your terminal.
  3. Could you provide the operating system you are using and its version?
  4. Could you provide the traceback of the error messages you are receiving?

This additional information will help me understand the context better and provide a more accurate solution.

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a ๐Ÿ‘ if this is helpful and ๐Ÿ‘Ž if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

@dosu-bot LangChain Version: 0.0.336 Python Version: 3.9.11 OS version: Ubuntu 18.04.6 LTS

https (CLOSE_WAIT)
python  23029 swapnil  179u  IPv4 3854749      0t0      TCP lightsaber.blr.datax.ai:57754->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  180u  IPv4 3854819      0t0      TCP lightsaber.blr.datax.ai:42772->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  181u  IPv4 3854839      0t0      TCP lightsaber.blr.datax.ai:36130->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  182u  IPv4 3854869      0t0      TCP lightsaber.blr.datax.ai:37582->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  183u  IPv4 3854881      0t0      TCP lightsaber.blr.datax.ai:35874->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  184u  IPv4 3854897      0t0      TCP lightsaber.blr.datax.ai:52328->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  185u  IPv4 3854916      0t0      TCP lightsaber.blr.datax.ai:39264->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  186u  IPv4 3854941      0t0      TCP lightsaber.blr.datax.ai:52046->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  187u  IPv4 3854960      0t0      TCP lightsaber.blr.datax.ai:34418->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  188u  IPv4 3855031      0t0      TCP lightsaber.blr.datax.ai:55208->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  189u  IPv4 3855053      0t0      TCP lightsaber.blr.datax.ai:55214->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  190u  IPv4 3855066      0t0      TCP lightsaber.blr.datax.ai:50402->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  191u  IPv4 3855108      0t0      TCP lightsaber.blr.datax.ai:42216->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  192u  IPv4 3866215      0t0      TCP lightsaber.blr.datax.ai:47614->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  193u  IPv4 3866435      0t0      TCP lightsaber.blr.datax.ai:57996->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  194u  IPv4 3855224      0t0      TCP lightsaber.blr.datax.ai:37806->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  195u  IPv4 3855253      0t0      TCP lightsaber.blr.datax.ai:51718->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  196u  IPv4 3855323      0t0      TCP lightsaber.blr.datax.ai:42224->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  197u  IPv4 3868706      0t0      TCP lightsaber.blr.datax.ai:40264->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  198u  IPv4 3868739      0t0      TCP lightsaber.blr.datax.ai:48818->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  199u  IPv4 3868790      0t0      TCP lightsaber.blr.datax.ai:56360->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  200u  IPv4 3868805      0t0      TCP lightsaber.blr.datax.ai:59018->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  201u  IPv4 3870493      0t0      TCP lightsaber.blr.datax.ai:56072->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  202u  IPv4 3869227      0t0      TCP lightsaber.blr.datax.ai:56586->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  203u  IPv4 3869283      0t0      TCP lightsaber.blr.datax.ai:34010->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  204u  IPv4 3869377      0t0      TCP lightsaber.blr.datax.ai:42904->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  205u  IPv4 3869401      0t0      TCP lightsaber.blr.datax.ai:52564->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  206u  IPv4 3869467      0t0      TCP lightsaber.blr.datax.ai:39588->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  207u  IPv4 3869482      0t0      TCP lightsaber.blr.datax.ai:43918->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  208u  IPv4 3869497      0t0      TCP lightsaber.blr.datax.ai:44350->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  209u  IPv4 3869516      0t0      TCP lightsaber.blr.datax.ai:34386->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  210u  IPv4 3869561      0t0      TCP lightsaber.blr.datax.ai:53858->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  211u  IPv4 3869578      0t0      TCP lightsaber.blr.datax.ai:45536->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  212u  IPv4 3869596      0t0      TCP lightsaber.blr.datax.ai:39700->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  213u  IPv4 3869621      0t0      TCP lightsaber.blr.datax.ai:54074->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  214u  IPv4 3869635      0t0      TCP lightsaber.blr.datax.ai:38162->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  215u  IPv4 3869650      0t0      TCP lightsaber.blr.datax.ai:52658->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  216u  IPv4 3869664      0t0      TCP lightsaber.blr.datax.ai:40312->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  217u  IPv4 3869680      0t0      TCP lightsaber.blr.datax.ai:57998->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  218u  IPv4 3882011      0t0      TCP lightsaber.blr.datax.ai:50194->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  219u  IPv4 3882062      0t0      TCP lightsaber.blr.datax.ai:55854->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  220u  IPv4 3882082      0t0      TCP lightsaber.blr.datax.ai:58646->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  221u  IPv4 3882096      0t0      TCP lightsaber.blr.datax.ai:55586->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  222u  IPv4 3882125      0t0      TCP lightsaber.blr.datax.ai:37348->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  223u  IPv4 3882166      0t0      TCP lightsaber.blr.datax.ai:49458->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  224u  IPv4 3886265      0t0      TCP lightsaber.blr.datax.ai:54214->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  225u  IPv4 3882225      0t0      TCP lightsaber.blr.datax.ai:60500->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  226u  IPv4 3882320      0t0      TCP lightsaber.blr.datax.ai:49820->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  227u  IPv4 3882333      0t0      TCP lightsaber.blr.datax.ai:52514->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  228u  IPv4 3882361      0t0      TCP lightsaber.blr.datax.ai:40122->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  229u  IPv4 3882376      0t0      TCP lightsaber.blr.datax.ai:50120->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  230u  IPv4 3881929      0t0      TCP lightsaber.blr.datax.ai:57992->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  231u  IPv4 3882434      0t0      TCP lightsaber.blr.datax.ai:46946->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  232u  IPv4 3882452      0t0      TCP lightsaber.blr.datax.ai:47414->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  233u  IPv4 3882544      0t0      TCP lightsaber.blr.datax.ai:42524->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  234u  IPv4 3882573      0t0      TCP lightsaber.blr.datax.ai:59338->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  235u  IPv4 3882585      0t0      TCP lightsaber.blr.datax.ai:47466->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  236u  IPv4 3882631      0t0      TCP lightsaber.blr.datax.ai:52398->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  237u  IPv4 3882664      0t0      TCP lightsaber.blr.datax.ai:56052->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  238u  IPv4 3882689      0t0      TCP lightsaber.blr.datax.ai:56308->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  239u  IPv4 3882732      0t0      TCP lightsaber.blr.datax.ai:55804->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  240u  IPv4 3889848      0t0      TCP lightsaber.blr.datax.ai:46276->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  241u  IPv4 3882808      0t0      TCP lightsaber.blr.datax.ai:50904->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  242u  IPv4 3882868      0t0      TCP lightsaber.blr.datax.ai:58800->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  243u  IPv4 3882901      0t0      TCP lightsaber.blr.datax.ai:49006->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  244u  IPv4 3882973      0t0      TCP lightsaber.blr.datax.ai:45404->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  245u  IPv4 3898370      0t0      TCP lightsaber.blr.datax.ai:45438->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  246u  IPv4 3898394      0t0      TCP lightsaber.blr.datax.ai:45340->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  247u  IPv4 3898412      0t0      TCP lightsaber.blr.datax.ai:49060->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  248u  IPv4 3898426      0t0      TCP lightsaber.blr.datax.ai:49076->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  249u  IPv4 3899711      0t0      TCP lightsaber.blr.datax.ai:51712->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  250u  IPv4 3898448      0t0      TCP lightsaber.blr.datax.ai:37028->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  251u  IPv4 3898497      0t0      TCP lightsaber.blr.datax.ai:37038->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  252u  IPv4 3898508      0t0      TCP lightsaber.blr.datax.ai:60988->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  253u  IPv4 3898542      0t0      TCP lightsaber.blr.datax.ai:41814->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  254u  IPv4 3902572      0t0      TCP lightsaber.blr.datax.ai:49780->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  255u  IPv4 3898633      0t0      TCP lightsaber.blr.datax.ai:49792->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  256u  IPv4 3898643      0t0      TCP lightsaber.blr.datax.ai:49508->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  257u  IPv4 3898673      0t0      TCP lightsaber.blr.datax.ai:57860->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  258u  IPv4 3898690      0t0      TCP lightsaber.blr.datax.ai:37842->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  259u  IPv4 3898701      0t0      TCP lightsaber.blr.datax.ai:58240->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  260u  IPv4 3898786      0t0      TCP lightsaber.blr.datax.ai:56460->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  261u  IPv4 3898794      0t0      TCP lightsaber.blr.datax.ai:60768->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  262u  IPv4 3898909      0t0      TCP lightsaber.blr.datax.ai:58820->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  263u  IPv4 3898920      0t0      TCP lightsaber.blr.datax.ai:40340->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  264u  IPv4 3898940      0t0      TCP lightsaber.blr.datax.ai:58328->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  265u  IPv4 3908436      0t0      TCP lightsaber.blr.datax.ai:36560->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  266u  IPv4 3899106      0t0      TCP lightsaber.blr.datax.ai:35564->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  267u  IPv4 3899186      0t0      TCP lightsaber.blr.datax.ai:51096->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  268u  IPv4 3899266      0t0      TCP lightsaber.blr.datax.ai:57430->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  269u  IPv4 3899308      0t0      TCP lightsaber.blr.datax.ai:57396->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  270u  IPv4 3899335      0t0      TCP lightsaber.blr.datax.ai:56422->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  271u  IPv4 3899371      0t0      TCP lightsaber.blr.datax.ai:35486->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  272u  IPv4 3912796      0t0      TCP lightsaber.blr.datax.ai:48172->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  273u  IPv4 3912832      0t0      TCP lightsaber.blr.datax.ai:53634->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  274u  IPv4 3908762      0t0      TCP lightsaber.blr.datax.ai:57814->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  275u  IPv4 3912886      0t0      TCP lightsaber.blr.datax.ai:46724->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  276u  IPv4 3912894      0t0      TCP lightsaber.blr.datax.ai:33912->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  277u  IPv4 3912906      0t0      TCP lightsaber.blr.datax.ai:34942->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  278u  IPv4 3912950      0t0      TCP lightsaber.blr.datax.ai:44658->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  279u  IPv4 3912971      0t0      TCP lightsaber.blr.datax.ai:51570->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  280u  IPv4 3912988      0t0      TCP lightsaber.blr.datax.ai:48534->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  281u  IPv4 3913075      0t0      TCP lightsaber.blr.datax.ai:41484->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  282u  IPv4 3913104      0t0      TCP lightsaber.blr.datax.ai:34596->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  283u  IPv4 3913133      0t0      TCP lightsaber.blr.datax.ai:57198->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  284u  IPv4 3920086      0t0      TCP lightsaber.blr.datax.ai:36238->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  285u  IPv4 3920100      0t0      TCP lightsaber.blr.datax.ai:38828->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  286u  IPv4 3913264      0t0      TCP lightsaber.blr.datax.ai:38836->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  287u  IPv4 3913309      0t0      TCP lightsaber.blr.datax.ai:51614->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  288u  IPv4 3913328      0t0      TCP lightsaber.blr.datax.ai:32778->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  289u  IPv4 3921521      0t0      TCP lightsaber.blr.datax.ai:37786->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  290u  IPv4 3913421      0t0      TCP lightsaber.blr.datax.ai:55770->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  291u  IPv4 3913443      0t0      TCP lightsaber.blr.datax.ai:59788->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  292u  IPv4 3913467      0t0      TCP lightsaber.blr.datax.ai:46852->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  293u  IPv4 3913493      0t0      TCP lightsaber.blr.datax.ai:54094->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  294u  IPv4 3913520      0t0      TCP lightsaber.blr.datax.ai:40886->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  295u  IPv4 3913537      0t0      TCP lightsaber.blr.datax.ai:51738->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  296u  IPv4 3913597      0t0      TCP lightsaber.blr.datax.ai:56894->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  297u  IPv4 3913641      0t0      TCP lightsaber.blr.datax.ai:42896->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  298u  IPv4 3913681      0t0      TCP lightsaber.blr.datax.ai:37550->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  299u  IPv4 3928174      0t0      TCP lightsaber.blr.datax.ai:59276->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  300u  IPv4 3930130      0t0      TCP lightsaber.blr.datax.ai:57388->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  301u  IPv4 3930139      0t0      TCP lightsaber.blr.datax.ai:48186->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  302u  IPv4 3930163      0t0      TCP lightsaber.blr.datax.ai:46914->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  303u  IPv4 3930202      0t0      TCP lightsaber.blr.datax.ai:56174->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  304u  IPv4 3930242      0t0      TCP lightsaber.blr.datax.ai:42198->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  305u  IPv4 3930263      0t0      TCP lightsaber.blr.datax.ai:33980->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  306u  IPv4 3930277      0t0      TCP lightsaber.blr.datax.ai:59400->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  307u  IPv4 3930317      0t0      TCP lightsaber.blr.datax.ai:44578->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  308u  IPv4 3930335      0t0      TCP lightsaber.blr.datax.ai:39342->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  309u  IPv4 3930363      0t0      TCP lightsaber.blr.datax.ai:44914->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  310u  IPv4 3930383      0t0      TCP lightsaber.blr.datax.ai:47150->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  311u  IPv4 3930399      0t0      TCP lightsaber.blr.datax.ai:50390->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  312u  IPv4 3930430      0t0      TCP lightsaber.blr.datax.ai:59138->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  313u  IPv4 3930453      0t0      TCP lightsaber.blr.datax.ai:45926->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  314u  IPv4 3938152      0t0      TCP lightsaber.blr.datax.ai:38484->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  315u  IPv4 3930568      0t0      TCP lightsaber.blr.datax.ai:50888->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  316u  IPv4 3930616      0t0      TCP lightsaber.blr.datax.ai:59658->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  317u  IPv4 3930632      0t0      TCP lightsaber.blr.datax.ai:51912->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  318u  IPv4 3930665      0t0      TCP lightsaber.blr.datax.ai:44136->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  319u  IPv4 3930712      0t0      TCP lightsaber.blr.datax.ai:36260->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  320u  IPv4 3930736      0t0      TCP lightsaber.blr.datax.ai:41378->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  321u  IPv4 3930775      0t0      TCP lightsaber.blr.datax.ai:33522->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  322u  IPv4 3930881      0t0      TCP lightsaber.blr.datax.ai:48728->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  323u  IPv4 3930920      0t0      TCP lightsaber.blr.datax.ai:38026->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  324u  IPv4 3930966      0t0      TCP lightsaber.blr.datax.ai:57126->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  325u  IPv4 3945597      0t0      TCP lightsaber.blr.datax.ai:45844->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  326u  IPv4 3931007      0t0      TCP lightsaber.blr.datax.ai:43596->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  327u  IPv4 3931032      0t0      TCP lightsaber.blr.datax.ai:52494->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  328u  IPv4 3931108      0t0      TCP lightsaber.blr.datax.ai:50772->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  329u  IPv4 3949573      0t0      TCP lightsaber.blr.datax.ai:44612->160.79.104.10:https (CLOSE_WAIT)
python  23029 swapnil  330u  IPv4 3949591      0t0      TCP lightsaber.blr.datax.ai:44122->160.79.104.10:https (CLOSE_WAIT)
dosubot[bot] commented 1 year ago

๐Ÿค–

I understand the issues you're facing with the CLOSE_WAIT error and the "too many files open" error. These issues are indeed related to the handling of a large number of connections. However, without the specific code snippet you're using, it's difficult to provide a precise solution.

To help you better, could you please provide the following additional information:

  1. The specific part of your code where you're opening and closing the connections.
  2. The method you're using to manage these connections.
  3. The type of connections you're handling (e.g., HTTP, TCP, etc.).
  4. Any error handling or exception management code you've implemented.

This information will help me understand your current implementation and provide a more accurate solution.

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a ๐Ÿ‘ if this is helpful and ๐Ÿ‘Ž if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

swapnil-lader commented 1 year ago

@dosu-bot

code where you're opening and closing the connections

def qa_chain():
    start_qa = time.time()
    qa_chain_output = StuffDocumentsChain(
        llm_chain=LLMChain(
            llm=ChatAnthropic(
                model_name="claude-2",
                temperature=0,
                top_p=1,
                max_tokens_to_sample=500000,
                default_request_timeout=30
            ),
            prompt=ChatPromptTemplate(
                messages=[
                    SystemMessage(content='You are a world class & knowledgeable product catalog assistance.'),
                    HumanMessagePromptTemplate.from_template('Context:\n{context}'),
                    HumanMessagePromptTemplate.from_template('{format_instructions}'),
                    HumanMessagePromptTemplate.from_template('Questions:\n{question}'),
                    HumanMessage(content='Tips: Make sure to answer in the correct format.'),
                ],
                partial_variables={"format_instructions": format_instructions}
            )
        ),
        document_variable_name='context',
        document_prompt=PromptTemplate(
            template='Content: {page_content}',
            input_variables=['page_content'],
        )
    )
    logger.info("Finished QA Chain in {}",time.time()-start_qa)
    return qa_chain_output

def generate(input_data, retriever):
    start_generate = time.time()
    doc_query = jaconv.z2h(
        jaconv.normalize(
            "ใ‚ณใƒผใƒ‰: {}\n่ฃฝๅ“ใฎ็จฎ้กž: {}\nใƒกใƒผใ‚ซใƒผๅ“ๅ: {}".format(
                str(input_data["Product Code"]),
                str(input_data["Product Type"]),
                str(input_data["Product Name"])
            ),
            "NFKC"
        ),
        kana=False,
        digit=True,
        ascii=True
    )
    docs = RecursiveCharacterTextSplitter(
        chunk_size=512,
        chunk_overlap=128
    ).split_documents(
        retriever.get_relevant_documents(doc_query)
    )
    docs = [
        doc
        for doc in docs
        if len(
            find_near_matches(
                str(input_data["Product Code"]),
                str(doc.page_content),
                max_l_dist=1
            )
        ) > 0
    ]
    pages = list(set([str(doc.metadata["page"]) for doc in docs]))

    question = (
        "Analyze the provided context and understand to extract the following attributes as precise as possible for {}"
        "Attributes:\n"
        "{}\n"
        "Return it in this JSON format: {}. "
        "if no information found or not sure return \"None\"."
    )
    generate_output = output_parser.parse(
        qa_chain().run({
            "input_documents": docs,
            "question": question.format(
                str(input_data["Product Code"]),
                '\n'.join([
                    f" {i + 1}. What is the value of \"{attrib}\"?"
                    for i, attrib in enumerate(input_data["Attributes"])
                ]),
                str({"answer": {attrib: f"value of {attrib}" for attrib in input_data["Attributes"]}}),
            )
        })
    )["answer"], ';'.join(pages)
    logger.info("Finished QA Chain in {}",time.time()-start_generate)
    return generate_output

method you're using to manage these connections. : ChatAnthropic

The type of connections you're handling : TCP

Any error handling or exception management code you've implemented:

try:
     generate()
except Exception as e:
     logger.info("Error at {},e)
dosubot[bot] commented 1 year ago

๐Ÿค–

Based on the information you've provided and the context of the LangChain repository, the CLOSE_WAIT and "too many files open" errors you're encountering are typically related to system-level configurations and not directly related to the LangChain framework or the Python code you're running. However, there are a few things you can do to mitigate these issues:

  1. Increase the maximum number of open files allowed by the system. You can do this by modifying the ulimit settings. You can check the current limit by running ulimit -n in the terminal. To increase the limit, you can use ulimit -n <new_limit>. Please note that this change is temporary and only applies to the current session. To make it permanent, you would need to modify the /etc/security/limits.conf file.

  2. Make sure you are properly closing any open connections in your code. If you're opening a file or a network connection, make sure to close it when you're done with it. In Python, it's a good practice to use the with statement when dealing with file operations, which automatically takes care of closing the file.

  3. The CLOSE_WAIT issue is usually related to the TCP connections not being closed properly. It means the local end of the connection has received a FIN from the other end, but the OS is waiting for the program at the local end to actually close its connection. The issue is usually either in the program (not closing connections) or in a firewall that is not allowing the FIN packet to reach the program. Check your firewall settings and make sure your program is properly closing its connections.

  4. If you're using a library or a third-party service that manages connections for you, make sure to check if there are any known issues or limitations with the service. In your case, you're using the OpenAI API, so you might want to check their documentation and see if there are any limitations or best practices you should follow. Also, ensure that the http_client in your LangChain configuration is properly managed and closed after use.

Please note that these are general suggestions and might not directly solve your issue. If the problem persists, you might want to consider getting help from a system administrator or a network expert.

As for the ChatAnthropic method in LangChain and how it handles TCP connections, I wasn't able to find specific information within the repository. It's possible that the answer may be available elsewhere or I could have missed it. If you have more specific information about how the ChatAnthropic method is implemented or used in your code, it could help in providing a more accurate solution.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a ๐Ÿ‘ if this is helpful and ๐Ÿ‘Ž if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

swapnil-lader commented 1 year ago

@dosu-bot how can close the connection created by ChatAnthropic LLM and LMChain in the above code before iterating to a new request?

dosubot[bot] commented 1 year ago

๐Ÿค–

To properly close the connection created by ChatAnthropic LLM and LMChain before iterating to a new request, you can modify the _call and _acall methods in the OpenLLM class to close the client connection after each request.

For the synchronous _call method, you can close the HTTP or gRPC client after the request is made:

def _call(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[CallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        try:
            import openllm
        except ImportError as e:
            raise ImportError(
                "Could not import openllm. Make sure to install it with "
                "'pip install openllm'."
            ) from e

        copied = copy.deepcopy(self.llm_kwargs)
        copied.update(kwargs)
        config = openllm.AutoConfig.for_model(
            self._identifying_params["model_name"], **copied
        )
        if self._client:
            res = self._client.generate(
                prompt, **config.model_dump(flatten=True)
            ).responses[0]
            self._client.close()  # Close the client connection
        else:
            assert self._runner is not None
            res = self._runner(prompt, **config.model_dump(flatten=True))
        if isinstance(res, dict) and "text" in res:
            return res["text"]
        elif isinstance(res, str):
            return res
        else:
            raise ValueError(
                "Expected result to be a dict with key 'text' or a string. "
                f"Received {res}"
            )

For the asynchronous _acall method, you can close the asynchronous HTTP client after the request is made:

async def _acall(
        self,
        prompt: str,
        stop: Optional[List[str]] = None,
        run_manager: Optional[AsyncCallbackManagerForLLMRun] = None,
        **kwargs: Any,
    ) -> str:
        try:
            import openllm
        except ImportError as e:
            raise ImportError(
                "Could not import openllm. Make sure to install it with "
                "'pip install openllm'."
            ) from e
        copied = copy.deepcopy(self.llm_kwargs)
        copied.update(kwargs)
        config = openllm.AutoConfig.for_model(
            self._identifying_params["model_name"], **copied
        )
        if self._client:
            async_client = openllm.client.AsyncHTTPClient(self.server_url)
            res = (
                await async_client.generate(prompt, **config.model_dump(flatten=True))
            ).responses[0]
            await async_client.close()  # Close the client connection
        else:
            assert self._runner is not None
            (
                prompt,
                generate_kwargs,
                postprocess_kwargs,
            ) = self._runner.llm.sanitize_parameters(prompt, **kwargs)
            generated_result = await self._runner.generate.async_run(
                prompt, **generate_kwargs
            )
            res = self._runner.llm.postprocess_generate(
                prompt, generated_result, **postprocess_kwargs
            )

        if isinstance(res, dict) and "text" in res:
            return res["text"]
        elif isinstance(res, str):
            return res
        else:
            raise ValueError(
                "Expected result to be a dict with key 'text' or a string. "
                f"Received {res}"
            )

Please note that the close method should be available in the client classes (HTTPClient, GrpcClient, AsyncHTTPClient) to close the connection. If these methods are not available, you may need to check the openllm library documentation or source code to find the correct method to close the connection.

You can find these methods in the openllm.py file in the llms directory of the LangChain repository. Here is the link to the file: libs/langchain/langchain/llms/openllm.py

I hope this helps! Let me know if you have any other questions.

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a ๐Ÿ‘ if this is helpful and ๐Ÿ‘Ž if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

snayan06 commented 9 months ago

@swapnil-lader were u able to resolve this ?? facing similar issue with openai client as well

swapnil-lader commented 9 months ago

@swapnil-lader were u able to resolve this ?? facing similar issue with openai client as well

Hey yes, it was a simple code-level mistake, We have to make sure to create a client outside the functions globally and use its global variable inside the lang chain. It resolved multiple connection issues for me.

snayan06 commented 9 months ago

@swapnil-lader so u made the chain global ? Can u share the code what changes u made.

Venkat2811 commented 8 months ago

Underlying anthropic python sdk could be the root cause for this. Similar issue occurred with openai lib.