search

finaldie / auto-news

A personal news aggregator to pull information from multi-sources + LLM (ChatGPT/Gemini/Ollama via LangChain) to help us reading efficiently with less noises, the sources including: Tweets, RSS, YouTube, Web Articles, Reddit, and personal Journal notes.
MIT License
517 stars 56 forks source link

Update langchain-community to version 0.3.1 #96

Closed xlzuvekas closed 2 months ago

xlzuvekas commented 2 months ago

Dependency Update:

Update langchain-community to version 0.3.1

https://github.com/langchain-ai/langchain/commit/28ad244e775610d4826fbfe34c62dba826cf26c9

Dependency Update:

Reason for Change

The application previously encountered the following error during runtime:


TypeError: unsupported operand type(s) for +=: 'dict' and 'dict'

This error originated from the langchain_community/chat_models/openai.py file, specifically within the _combine_llm_outputs method. The root cause was the unsupported operation of adding two dictionaries using the += operator. Updating langchain to version 0.3.1 introduces support for nested dictionaries.

Logs:

2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO - Traceback (most recent call last):
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/opt/***/run/auto-news/src/af_save.py", line 259, in <module>
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     run(args)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/opt/***/run/auto-news/src/af_save.py", line 240, in run
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     stat = process_rss(args)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/opt/***/run/auto-news/src/af_save.py", line 162, in process_rss
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     data_summarized = op.summarize(data_filtered)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/opt/***/run/auto-news/src/ops_rss.py", line 298, in summarize
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     summary = llm_agent.run(content)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -               ^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/opt/***/run/auto-news/src/llm_agent.py", line 355, in run
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     summary_resp = self.llmchain.run(docs)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -                    ^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain_core/_api/deprecation.py", line 145, in warning_emitting_wrapper
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     return wrapped(*args, **kwargs)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/base.py", line 538, in run
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain_core/_api/deprecation.py", line 145, in warning_emitting_wrapper
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     return wrapped(*args, **kwargs)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/base.py", line 363, in __call__
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     return self.invoke(
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/base.py", line 162, in invoke
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     raise e
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/base.py", line 156, in invoke
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     self._call(inputs, run_manager=run_manager)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/combine_documents/base.py", line 136, in _call
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     output, extra_return_dict = self.combine_docs(
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -                                 ^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/combine_documents/map_reduce.py", line 225, in combine_docs
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     map_results = self.llm_chain.apply(
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -                   ^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/llm.py", line 227, in apply
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     raise e
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/llm.py", line 224, in apply
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     response = self.generate(input_list, run_manager=run_manager)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain/chains/llm.py", line 115, in generate
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     return self.llm.generate_prompt(
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 544, in generate_prompt
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     return self.generate(prompt_messages, stop=stop, callbacks=callbacks, **kwargs)
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain_core/language_models/chat_models.py", line 413, in generate
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     llm_output = self._combine_llm_outputs([res.llm_output for res in results])
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -   File "/home/***/.local/lib/python3.11/site-packages/langchain_community/chat_models/openai.py", line 377, in _combine_llm_outputs
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO -     overall_token_usage[k] += v
[2024-09-26, 17:13:28 UTC] {subprocess.py:93} INFO - TypeError: unsupported operand type(s) for +=: 'dict' and 'dict'
[2024-09-26, 17:13:29 UTC] {subprocess.py:97} INFO - Command exited with return code 1
[2024-09-26, 17:13:29 UTC] {taskinstance.py:2731} ERROR - Task failed with exception

Error Type:

TypeError

Error Message:

unsupported operand type(s) for +=: 'dict' and 'dict'

Location:

langchain_community/chat_models/openai.py, line 377, in _combine_llm_outputs

Description:

An attempt was made to use the += operator to add two dictionaries (overall_token_usage[k] and v), which is not supported in Python. This error occurs within the langchain_community package while processing the output of language model calls.

Call Stack Overview:

  1. User Script:
    • File: /opt/***/run/auto-news/src/af_save.py
    • Function: run → process_rss → op.summarize
  2. Langchain Library:
    • File: /home/***/.local/lib/python3.11/site-packages/langchain/chains/base.py
    • Function: run → __call__ → invoke
  3. LLM Agent:
    • File: /home/***/.local/lib/python3.11/site-packages/langchain/chains/combine_documents/map_reduce.py
    • Function: combine_docs → self.llm_chain.apply
  4. Final Call Leading to Error:
    • File: /home/***/.local/lib/python3.11/site-packages/langchain_community/chat_models/openai.py
    • Function: _combine_llm_outputs

Impact:

The task failed to execute successfully, resulting in the termination of the process with a return code 1. This affects the auto-news application's ability to process and summarize RSS feeds.

finaldie commented 2 months ago

@xlzuvekas , super thanks for addressing the issue along with the details logs for the investigation! I'll merge it and create a new tag.

xlzuvekas commented 2 months ago

@xlzuvekas , super thanks for addressing the issue along with the details logs for the investigation! I'll merge it and create a new tag.

Happy to help! Cheers!