search

langchain-ai / langchain

🦜🔗 Build context-aware reasoning applications
https://python.langchain.com
MIT License
93.02k stars 14.94k forks source link

Error run or invoke method of QAGenerationChain #20406

Closed sriakhil25 closed 2 months ago

sriakhil25 commented 5 months ago

Checked other resources

Example Code

Hi I am trying to generate QA from a pdf

Running following code snippet -

text_splitter = RecursiveCharacterTextSplitter(

Set a really small chunk size, just to show.

chunk_size = 1000,
chunk_overlap  = 100,
length_function = len,

)

loader = PyPDFLoader(data_pdf) pages = loader.load_and_split(text_splitter=text_splitter)

len(pages) #260

templ = """You are a smart assistant designed to come up with meaninful question and answer pair. The question should be to the point and the answer should be as detailed as possible. Given a piece of text, you must come up with a question and answer pair that can be used to evaluate a QA bot. Do not make up stuff. Stick to the text to come up with the question and answer pair. When coming up with this question/answer pair, you must respond in the following format:

{{
    "question": "$YOUR_QUESTION_HERE",
    "answer": "$THE_ANSWER_HERE"
}}

Everything between the ``` must be valid json.

Please come up with a question/answer pair, in the specified JSON format, for the following text:

{text}"""

PROMPT = PromptTemplate.from_template(templ)

llm = Cohere(model="command", temperature=0) # command, command-light chain = QAGenerationChain.from_llm(llm=llm, prompt=PROMPT)

llm is as follows

Cohere(client=<cohere.client.Client object at 0x00000188268D3BD0>, async_client=<cohere.client.AsyncClient object at ##0x0000018827B22690>, model='command', temperature=0.0)

By running following code I am expecting QA set to be generated but I am getting error

chain.invoke(pages[40].page_content)

I tried other models like OpenAI and Google Gemini Pro and QAGeneration chain fails at same step

Error Message and Stack Trace (if applicable)

AttributeError Traceback (most recent call last) Cell In[17], line 1 ----> 1 chain.invoke(pages[40].page_content)

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain\chains\base.py:163, in Chain.invoke(self, input, config, **kwargs) 161 except BaseException as e: 162 run_manager.on_chain_error(e) --> 163 raise e 164 run_manager.on_chain_end(outputs) 166 if include_run_info:

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain\chains\base.py:153, in Chain.invoke(self, input, config, **kwargs) 150 try: 151 self._validate_inputs(inputs) 152 outputs = ( --> 153 self._call(inputs, run_manager=run_manager) 154 if new_arg_supported 155 else self._call(inputs) 156 ) 158 final_outputs: Dict[str, Any] = self.prep_outputs( 159 inputs, outputs, return_only_outputs 160 ) 161 except BaseException as e:

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain\chains\qa_generation\base.py:73, in QAGenerationChain._call(self, inputs, run_manager) 67 def _call( 68 self, 69 inputs: Dict[str, Any], 70 run_manager: Optional[CallbackManagerForChainRun] = None, 71 ) -> Dict[str, List]: 72 docs = self.text_splitter.create_documents([inputs[self.input_key]]) ---> 73 results = self.llm_chain.generate( 74 [{"text": d.page_content} for d in docs], run_manager=run_manager 75 ) 76 qa = [json.loads(res[0].text) for res in results.generations] 77 return {self.output_key: qa}

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain\chains\llm.py:115, in LLMChain.generate(self, input_list, run_manager) 113 callbacks = run_manager.get_child() if run_manager else None 114 if isinstance(self.llm, BaseLanguageModel): --> 115 return self.llm.generate_prompt( 116 prompts, 117 stop, 118 callbacks=callbacks, 119 self.llm_kwargs, 120 ) 121 else: 122 results = self.llm.bind(stop=stop, self.llm_kwargs).batch( 123 cast(List, prompts), {"callbacks": callbacks} 124 )

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_core\language_models\llms.py:597, in BaseLLM.generate_prompt(self, prompts, stop, callbacks, kwargs) 589 def generate_prompt( 590 self, 591 prompts: List[PromptValue], (...) 594 kwargs: Any, 595 ) -> LLMResult: 596 prompt_strings = [p.to_string() for p in prompts] --> 597 return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_core\language_models\llms.py:767, in BaseLLM.generate(self, prompts, stop, callbacks, tags, metadata, run_name, run_id, kwargs) 752 if (self.cache is None and get_llm_cache() is None) or self.cache is False: 753 run_managers = [ 754 callback_manager.on_llm_start( 755 dumpd(self), (...) 765 ) 766 ] --> 767 output = self._generate_helper( 768 prompts, stop, run_managers, bool(new_arg_supported), kwargs 769 ) 770 return output 771 if len(missing_prompts) > 0:

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_core\language_models\llms.py:634, in BaseLLM._generate_helper(self, prompts, stop, run_managers, new_arg_supported, **kwargs) 632 for run_manager in run_managers: 633 run_manager.on_llm_error(e, response=LLMResult(generations=[])) --> 634 raise e 635 flattened_outputs = output.flatten() 636 for manager, flattened_output in zip(run_managers, flattened_outputs):

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_core\language_models\llms.py:621, in BaseLLM._generate_helper(self, prompts, stop, run_managers, new_arg_supported, kwargs) 611 def _generate_helper( 612 self, 613 prompts: List[str], (...) 617 kwargs: Any, 618 ) -> LLMResult: 619 try: 620 output = ( --> 621 self._generate( 622 prompts, 623 stop=stop, 624 # TODO: support multiple run managers 625 run_manager=run_managers[0] if run_managers else None, 626 **kwargs, 627 ) 628 if new_arg_supported 629 else self._generate(prompts, stop=stop) 630 ) 631 except BaseException as e: 632 for run_manager in run_managers:

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_core\language_models\llms.py:1231, in LLM._generate(self, prompts, stop, run_manager, kwargs) 1228 new_arg_supported = inspect.signature(self._call).parameters.get("run_manager") 1229 for prompt in prompts: 1230 text = ( -> 1231 self._call(prompt, stop=stop, run_manager=run_manager, kwargs) 1232 if new_arg_supported 1233 else self._call(prompt, stop=stop, **kwargs) 1234 ) 1235 generations.append([Generation(text=text)]) 1236 return LLMResult(generations=generations)

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_community\llms\cohere.py:217, in Cohere._call(self, prompt, stop, run_manager, kwargs) 202 """Call out to Cohere's generate endpoint. 203 204 Args: (...) 214 response = cohere("Tell me a joke.") 215 """ 216 params = self._invocation_params(stop, kwargs) --> 217 response = completion_with_retry( 218 self, model=self.model, prompt=prompt, **params 219 ) 220 _stop = params.get("stop_sequences") 221 return self._process_response(response, _stop)

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_community\llms\cohere.py:45, in completion_with_retry(llm, kwargs) 43 def completion_with_retry(llm: Cohere, kwargs: Any) -> Any: 44 """Use tenacity to retry the completion call.""" ---> 45 retry_decorator = _create_retry_decorator(llm) 47 @retry_decorator 48 def _completion_with_retry(kwargs: Any) -> Any: 49 return llm.client.generate(kwargs)

File ~\anaconda3\envs\GenAI_py311\Lib\site-packages\langchain_community\llms\cohere.py:38, in _create_retry_decorator(llm) 31 max_seconds = 10 32 # Wait 2^x * 1 second between each retry starting with 33 # 4 seconds, then up to 10 seconds, then 10 seconds afterwards 34 return retry( 35 reraise=True, 36 stop=stop_after_attempt(llm.max_retries), 37 wait=wait_exponential(multiplier=1, min=min_seconds, max=max_seconds), ---> 38 retry=(retry_if_exception_type(cohere.error.CohereError)), 39 before_sleep=before_sleep_log(logger, logging.WARNING), 40 )

AttributeError: module 'cohere' has no attribute 'error'

Description

chain = QAGenerationChain.from_llm(llm=llm, prompt=PROMPT) chain.invoke(pages[40].page_content)

Chain.run and chain.invoke giving error

System Info

python 3.11 langchain==0.1.9 langchain-community==0.0.24 langchain-core==0.1.42 langchain-google-genai==1.0.2

sriakhil25 commented 5 months ago

Issue resolved after upgrading

jhu2004 commented 5 months ago

Issue resolved after upgrading

hey, can I ask what did you upgrade to resolve this? I seem to have the same issue. thanks.

sriakhil25 commented 5 months ago

Issue resolved after upgrading

hey, can I ask what did you upgrade to resolve this? I seem to have the same issue. thanks.

Yes this issue is resolved for me after upgrade