search

opea-project / GenAIComps

GenAI components at micro-service level; GenAI service composer to create mega-service
Apache License 2.0
61 stars 122 forks source link

[Agent] comps/agent /langchain/ validate service fails #805

Open xiguiw opened 2 days ago

xiguiw commented 2 days ago

Follow agent comp started the TGI and Agent service, agent service failed.

start agent microservices with tgi

TGI service works, however, validate agent service failed.

Set the Environment

export ip_address=$(hostname -I | awk '{print $1}')
export model="Intel/neural-chat-7b-v3-3"
export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
  1. start TGI service (with No Guadi device, run TGI on Xeon): docker run -d -it --name "comps-tgi-xeon-service" -p 8080:80 -v ./data:/data -e HF_TOKEN=$HF_TOKEN -e HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} -e http_proxy=${http_proxy} -e https_proxy=${https_proxy} -e OMPI_MCA_btl_vader_single_copy_mechanism=none --cap-add=sys_nice --ipc=host -e no_proxy=${no_proxy} ghcr.io/huggingface/text-generation-inference:sha-e4201f4-intel-cpu --model-id $model --max-input-tokens 4096 --max-total-tokens 8092 --cuda-graphs 0

1.1 validate TGI service works 

 curl http://${ip_address}:8080/generate \
      -X POST \
      -d '{"inputs":"What is Deep Learning?","parameters":{"max_new_tokens":17, "do_sample": true}}' \
      -H 'Content-Type: application/json'

output correct:

{"generated_text":"\n\nDeep Learning is a subset of Machine Learning. It basically builds algorithms that try"}
  1. Start agent service docker image: opea/agent-langchain:latest (v1.0 pull from docker hub) 
    # Agent
docker run -d --runtime=runc --name="comps-langchain-agent-endpoint" -v $WORKPATH/comps/agent/langchain/tools:/home/user/comps/agent/langchain/tools -p 9090:9090 --ipc=host -e HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN} -e model=${model} -e ip_address=${ip_address} -e strategy=react_langchain -e llm_endpoint_url=http://${ip_address}:8080 -e llm_engine=tgi -e recursion_limit=5 -e require_human_feedback=false -e tools=/home/user/comps/agent/langchain/tools/custom_tools.yaml opea/agent-langchain:latest

check status

docker logs comps-langchain-agent-endpoint

docker logs comps-langchain-agent-endpoint
/home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name_or_path" in Audio2TextDoc has conflict with protected namespace "model_".

You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
  warnings.warn(
[2024-10-21 04:00:37,249] [    INFO] - Base service - CORS is enabled.
[2024-10-21 04:00:37,250] [    INFO] - Base service - Setting up HTTP server
[2024-10-21 04:00:37,250] [    INFO] - Base service - Uvicorn server setup on port 9090
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:9090 (Press CTRL+C to quit)
[2024-10-21 04:00:37,258] [    INFO] - Base service - HTTP server setup successful

The following Validate Microservice fails

curl http://${ip_address}:9090/v1/chat/completions -X POST -H "Content-Type: application/json" -d '{
     "query": "What is the weather today in Austin?"
    }'

expected output

data: 'The temperature in Austin today is 78°F.</s>'

data: [DONE]

Actual result:

Note HUGGINGFACEHUB_API_TOKEN is set and used in first TGI services. But the log to check HUGGINGFACEHUB_API_TOKEN.

(2024-10-21T04:10:05.346595464Z pydantic_core._pydantic_core.ValidationError: 1 validation error for HuggingFaceEndpoint 2024-10-21T04:10:05.346602020Z Value error, Could not authenticate with huggingface_hub. Please check your API token. [type=value_error, input_value={'endpoint_url': 'http://...://10.239.182.158:8080'}, input_type=dict] 2024-10-21T04:10:05.346611397Z For further information visit https://errors.pydantic.dev/2.9/v/value_error )

2024-10-21T04:00:36.995669826Z /home/user/.local/lib/python3.11/site-packages/pydantic/_internal/_fields.py:132: UserWarning: Field "model_name_or_path" in Audio2TextDoc has conflict with protected namespace "model_".
2024-10-21T04:00:36.995795632Z
2024-10-21T04:00:36.995803681Z You may be able to resolve this warning by setting `model_config['protected_namespaces'] = ()`.
2024-10-21T04:00:36.995808817Z   warnings.warn(
2024-10-21T04:00:37.250048725Z [2024-10-21 04:00:37,249] [    INFO] - Base service - CORS is enabled.
2024-10-21T04:00:37.250983665Z [2024-10-21 04:00:37,250] [    INFO] - Base service - Setting up HTTP server
2024-10-21T04:00:37.251048112Z [2024-10-21 04:00:37,250] [    INFO] - Base service - Uvicorn server setup on port 9090
2024-10-21T04:00:37.258530142Z INFO:     Waiting for application startup.
2024-10-21T04:00:37.259042125Z INFO:     Application startup complete.
2024-10-21T04:00:37.259088323Z INFO:     Uvicorn running on http://0.0.0.0:9090 (Press CTRL+C to quit)
2024-10-21T04:00:37.259096032Z [2024-10-21 04:00:37,258] [    INFO] - Base service - HTTP server setup successful
2024-10-21T04:01:03.207863810Z /home/user/comps/agent/langchain/src/strategy/react/planner.py:6: LangChainDeprecationWarning: Importing ChatMessageHistory from langchain.memory is deprecated. Please replace deprecated imports:
2024-10-21T04:01:03.208040048Z
2024-10-21T04:01:03.208054538Z >> from langchain.memory import ChatMessageHistory
2024-10-21T04:01:03.208061913Z
2024-10-21T04:01:03.208067286Z with new imports of:
2024-10-21T04:01:03.208073018Z
2024-10-21T04:01:03.208078195Z >> from langchain_community.chat_message_histories import ChatMessageHistory
2024-10-21T04:01:03.208085274Z You can use the langchain cli to **automatically** upgrade many imports. Please see documentation here <https://python.langchain.com/v0.2/docs/versions/v0_2/>
2024-10-21T04:01:03.208093875Z   from langchain.memory import ChatMessageHistory
2024-10-21T04:01:03.476705209Z /home/user/comps/agent/langchain/src/strategy/base_agent.py:6: LangChainDeprecationWarning: As of langchain-core 0.3.0, LangChain uses pydantic v2 internally. The langchain.pydantic_v1 module was a compatibility shim for pydantic v1, and should no longer be used. Please update the code to import from Pydantic directly.
2024-10-21T04:01:03.476760777Z
2024-10-21T04:01:03.476769082Z For example, replace imports like: `from langchain.pydantic_v1 import BaseModel`
2024-10-21T04:01:03.476775537Z with: `from pydantic import BaseModel`
2024-10-21T04:01:03.476781159Z or the v1 compatibility namespace if you are working in a code base that has not been fully upgraded to pydantic 2 yet.      from pydantic.v1 import BaseModel
2024-10-21T04:01:03.476788679Z
2024-10-21T04:01:03.476794819Z   from ..tools import get_tools_descriptions
2024-10-21T04:10:05.333290359Z The token has not been saved to the git credentials helper. Pass `add_to_git_credential=True` in this function directly or `--add-to-git-credential` if using via `huggingface-cli` if you want to set the git credential as well.
2024-10-21T04:10:05.333424504Z INFO:     10.239.182.158:58208 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
2024-10-21T04:10:05.346211095Z ERROR:    Exception in ASGI application
2024-10-21T04:10:05.346263775Z Traceback (most recent call last):
2024-10-21T04:10:05.346270588Z   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
2024-10-21T04:10:05.346276128Z     result = await app(  # type: ignore[func-returns-value]
2024-10-21T04:10:05.346280250Z              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346284535Z   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
2024-10-21T04:10:05.346289968Z     return await self.app(scope, receive, send)
2024-10-21T04:10:05.346294046Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346298000Z   File "/home/user/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-10-21T04:10:05.346302406Z     await super().__call__(scope, receive, send)
2024-10-21T04:10:05.346306555Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/applications.py", line 113, in __call__
2024-10-21T04:10:05.346310970Z     await self.middleware_stack(scope, receive, send)
2024-10-21T04:10:05.346315266Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
2024-10-21T04:10:05.346320215Z     raise exc
2024-10-21T04:10:05.346324275Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
2024-10-21T04:10:05.346328817Z     await self.app(scope, receive, _send)
2024-10-21T04:10:05.346332748Z   File "/home/user/.local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 174, in __call__
2024-10-21T04:10:05.346337226Z     raise exc
2024-10-21T04:10:05.346340935Z   File "/home/user/.local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__
2024-10-21T04:10:05.346345357Z     await self.app(scope, receive, send_wrapper)
2024-10-21T04:10:05.346349190Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
2024-10-21T04:10:05.346353630Z     await self.app(scope, receive, send)
2024-10-21T04:10:05.346357847Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2024-10-21T04:10:05.346362326Z     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-10-21T04:10:05.346366610Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
2024-10-21T04:10:05.346371006Z     raise exc
2024-10-21T04:10:05.346374570Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
2024-10-21T04:10:05.346393075Z     await app(scope, receive, sender)
2024-10-21T04:10:05.346399606Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 715, in __call__
2024-10-21T04:10:05.346404820Z     await self.middleware_stack(scope, receive, send)
2024-10-21T04:10:05.346409348Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 735, in app
2024-10-21T04:10:05.346413670Z     await route.handle(scope, receive, send)
2024-10-21T04:10:05.346417730Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
2024-10-21T04:10:05.346422257Z     await self.app(scope, receive, send)
2024-10-21T04:10:05.346426226Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
2024-10-21T04:10:05.346430560Z     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-10-21T04:10:05.346434768Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
2024-10-21T04:10:05.346439237Z     raise exc
2024-10-21T04:10:05.346442897Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
2024-10-21T04:10:05.346447270Z     await app(scope, receive, sender)
2024-10-21T04:10:05.346451117Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 73, in app
2024-10-21T04:10:05.346455497Z     response = await f(request)
2024-10-21T04:10:05.346459362Z                ^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346463128Z   File "/home/user/.local/lib/python3.11/site-packages/fastapi/routing.py", line 301, in app
2024-10-21T04:10:05.346467375Z     raw_response = await run_endpoint_function(
2024-10-21T04:10:05.346471286Z                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346475237Z   File "/home/user/.local/lib/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
2024-10-21T04:10:05.346479615Z     return await dependant.call(**values)
2024-10-21T04:10:05.346483437Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346487790Z   File "/home/user/comps/agent/langchain/agent.py", line 56, in llm_generate
2024-10-21T04:10:05.346492660Z     agent_inst = instantiate_agent(args, args.strategy)
2024-10-21T04:10:05.346497006Z                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346501390Z   File "/home/user/comps/agent/langchain/src/agent.py", line 9, in instantiate_agent
2024-10-21T04:10:05.346506035Z     return ReActAgentwithLangchain(args, with_memory)
2024-10-21T04:10:05.346509928Z            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346514275Z   File "/home/user/comps/agent/langchain/src/strategy/react/planner.py", line 22, in __init__
2024-10-21T04:10:05.346518568Z     super().__init__(args)
2024-10-21T04:10:05.346526920Z   File "/home/user/comps/agent/langchain/src/strategy/base_agent.py", line 12, in __init__
2024-10-21T04:10:05.346531440Z     self.llm_endpoint = setup_llm(args)
2024-10-21T04:10:05.346536064Z                         ^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346539995Z   File "/home/user/comps/agent/langchain/src/utils.py", line 83, in setup_llm
2024-10-21T04:10:05.346544150Z     model = setup_hf_tgi_client(args)
2024-10-21T04:10:05.346548030Z             ^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346551910Z   File "/home/user/comps/agent/langchain/src/utils.py", line 40, in setup_hf_tgi_client
2024-10-21T04:10:05.346556524Z     llm = HuggingFaceEndpoint(
2024-10-21T04:10:05.346560804Z           ^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346564875Z   File "/home/user/.local/lib/python3.11/site-packages/langchain_core/load/serializable.py", line 112, in __init__
2024-10-21T04:10:05.346569788Z     super().__init__(*args, **kwargs)
2024-10-21T04:10:05.346573650Z   File "/home/user/.local/lib/python3.11/site-packages/pydantic/main.py", line 212, in __init__
2024-10-21T04:10:05.346577895Z     validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
2024-10-21T04:10:05.346583626Z                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-21T04:10:05.346595464Z pydantic_core._pydantic_core.ValidationError: 1 validation error for HuggingFaceEndpoint
2024-10-21T04:10:05.346602020Z   Value error, Could not authenticate with huggingface_hub. Please check your API token. [type=value_error, input_value={'endpoint_url': 'http://...://10.239.182.158:8080'}, input_type=dict]
2024-10-21T04:10:05.346611397Z     For further information visit https://errors.pydantic.dev/2.9/v/value_error
xuechendi commented 1 day ago

@xiguiw , I see the error logs said you didn't provide huggingface token:

Value error, Could not authenticate with huggingface_hub. Please check your API token.

Please make sure you have specified below env: HF_TOKEN="xxxxxxxxxxxxx" HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN}

xiguiw commented 1 day ago

@xiguiw , I see the error logs said you didn't provide huggingface token:

Value error, Could not authenticate with huggingface_hub. Please check your API token.

Please make sure you have specified below env: HF_TOKEN="xxxxxxxxxxxxx" HUGGINGFACEHUB_API_TOKEN=${HF_TOKEN}

Yes I found the log messages. The fact is huggingface token is set.

Here is what I set

Set the Environment

export ip_address=$(hostname -I | awk '{print $1}')
export model="Intel/neural-chat-7b-v3-3"
export HUGGINGFACEHUB_API_TOKEN=${HUGGINGFACEHUB_API_TOKEN}
export HF_TOKEN=${HUGGINGFACEHUB_API_TOKEN}

I wonder the "huggingface token" I set it part of the input. image

xuechendi commented 1 day ago

@xiguiw , may you also check the endpoint you provided, I saw logs show something as below

[type=value_error, input_value={'endpoint_url': 'http://...://10.239.182.158:8080'}

in 

2024-10-21T04:10:05.346595464Z pydantic_core._pydantic_core.ValidationError: 1 validation error for HuggingFaceEndpoint
2024-10-21T04:10:05.346602020Z   Value error, Could not authenticate with huggingface_hub. Please check your API token. [type=value_error, input_value={'endpoint_url': 'http://...://10.239.182.158:8080'}, input_type=dict]
2024-10-21T04:10:05.346611397Z     For further information visit https://errors.pydantic.dev/2.9/v/value_error
xiguiw commented 10 hours ago

@xuechendi Thanks Chendi,

it seems agent cannot access local TGI services (the llm endpoint) . After I added ip_address to the no_proxy environment and set no_proxy to the agent. it can access the TGI services.

But it raised another error this time:
Ratelimit. duckduckgo_search.exceptions.RatelimitException: https://duckduckgo.com/ 202 Ratelimit

2024-10-23T01:34:13.742835233Z > Entering new AgentExecutor chain...
2024-10-23T01:34:13.742842816Z ---
2024-10-23T01:34:13.742848336Z  Knowing the weather helps us plan our day
2024-10-23T01:34:13.742854873Z Action: duckduckgo_search
2024-10-23T01:34:13.742860706Z Action Input: 'weather today in Austin'
2024-10-23T01:34:13.742866428Z ObservationThe token has not been saved to the git credentials helper. Pass `add_to_git_credential=True` in this function directly or `--add-to-git-credential` if using via `huggingface-cli` if you want to set the git credential as well.
2024-10-23T01:34:13.742875048Z Token is valid (permission: read).
2024-10-23T01:34:13.742880653Z Your token has been saved to /home/user/.cache/huggingface/token
2024-10-23T01:34:13.742886173Z Login successful
2024-10-23T01:34:13.742891724Z [DuckDuckGoSearchRun(api_wrapper=DuckDuckGoSearchAPIWrapper(region='wt-wt', safesearch='moderate', time='y', max_results=5, backend='api', source='text'))]
2024-10-23T01:34:13.742897976Z -----------STREAMING-------------
2024-10-23T01:34:13.742903408Z INFO:     10.239.182.158:60956 - "POST /v1/chat/completions HTTP/1.1" 200 OK
2024-10-23T01:34:13.760912905Z /home/user/.local/lib/python3.11/site-packages/huggingface_hub/inference/_generated/_async_client.py:2091: FutureWarning: `stop_sequences` is a deprecated argument for `text_generation` task and will be removed in version '0.28.0'. Use `stop` instead.
2024-10-23T01:34:13.761000314Z   warnings.warn(
2024-10-23T01:34:19.292844549Z ERROR:    Exception in ASGI application
2024-10-23T01:34:19.292909574Z Traceback (most recent call last):
2024-10-23T01:34:19.292917586Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 257, in __call__
2024-10-23T01:34:19.292924780Z     await wrap(partial(self.listen_for_disconnect, receive))
2024-10-23T01:34:19.293049607Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 253, in wrap
2024-10-23T01:34:19.293056358Z     await func()
2024-10-23T01:34:19.293061949Z   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 230, in listen_for_disconnect
2024-10-23T01:34:19.293068815Z     message = await receive()
2024-10-23T01:34:19.293074145Z               ^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293080118Z   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 534, in receive
2024-10-23T01:34:19.293086675Z     await self.message_event.wait()
2024-10-23T01:34:19.293092365Z   File "/usr/local/lib/python3.11/asyncio/locks.py", line 213, in wait
2024-10-23T01:34:19.293098129Z     await fut
2024-10-23T01:34:19.293103345Z asyncio.exceptions.CancelledError: Cancelled by cancel scope 7c96024c43d0
2024-10-23T01:34:19.293109309Z
2024-10-23T01:34:19.293114734Z During handling of the above exception, another exception occurred:
2024-10-23T01:34:19.293120429Z
2024-10-23T01:34:19.293125825Z   + Exception Group Traceback (most recent call last):
2024-10-23T01:34:19.293131369Z   |   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 406, in run_asgi
2024-10-23T01:34:19.293137818Z   |     result = await app(  # type: ignore[func-returns-value]
2024-10-23T01:34:19.293143320Z   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293149115Z   |   File "/home/user/.local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
2024-10-23T01:34:19.293155025Z   |     return await self.app(scope, receive, send)
2024-10-23T01:34:19.293160354Z   |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293165735Z   |   File "/home/user/.local/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
2024-10-23T01:34:19.293171625Z   |     await super().__call__(scope, receive, send)
2024-10-23T01:34:19.293177015Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/applications.py", line 113, in __call__
2024-10-23T01:34:19.293182822Z   |     await self.middleware_stack(scope, receive, send)
2024-10-23T01:34:19.293208194Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 187, in __call__
2024-10-23T01:34:19.293213247Z   |     raise exc
2024-10-23T01:34:19.293217027Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
2024-10-23T01:34:19.293223629Z   |     await self.app(scope, receive, _send)
2024-10-23T01:34:19.293227785Z   |   File "/home/user/.local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 174, in __call__
2024-10-23T01:34:19.293232389Z   |     raise exc
2024-10-23T01:34:19.293236187Z   |   File "/home/user/.local/lib/python3.11/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 172, in __call__
2024-10-23T01:34:19.293240729Z   |     await self.app(scope, receive, send_wrapper)
2024-10-23T01:34:19.293244665Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
2024-10-23T01:34:19.293249000Z   |     await self.app(scope, receive, send)
2024-10-23T01:34:19.293253020Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
2024-10-23T01:34:19.293257655Z   |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
2024-10-23T01:34:19.293262034Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
2024-10-23T01:34:19.293267138Z   |     raise exc
2024-10-23T01:34:19.293270780Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
2024-10-23T01:34:19.293275318Z   |     await app(scope, receive, sender)
2024-10-23T01:34:19.293279280Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 715, in __call__
2024-10-23T01:34:19.293284329Z   |     await self.middleware_stack(scope, receive, send)
2024-10-23T01:34:19.293288214Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 735, in app
2024-10-23T01:34:19.293293005Z   |     await route.handle(scope, receive, send)
2024-10-23T01:34:19.293296998Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 288, in handle
2024-10-23T01:34:19.293301338Z   |     await self.app(scope, receive, send)
2024-10-23T01:34:19.293305255Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 76, in app
2024-10-23T01:34:19.293309540Z   |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
2024-10-23T01:34:19.293313935Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 62, in wrapped_app
2024-10-23T01:34:19.293318334Z   |     raise exc
2024-10-23T01:34:19.293321915Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/_exception_handler.py", line 51, in wrapped_app
2024-10-23T01:34:19.293331895Z   |     await app(scope, receive, sender)
2024-10-23T01:34:19.293336018Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/routing.py", line 74, in app
2024-10-23T01:34:19.293340755Z   |     await response(scope, receive, send)
2024-10-23T01:34:19.293344635Z   |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 250, in __call__
2024-10-23T01:34:19.293349038Z   |     async with anyio.create_task_group() as task_group:
2024-10-23T01:34:19.293353009Z   |   File "/home/user/.local/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 685, in __aexit__
2024-10-23T01:34:19.293357307Z   |     raise BaseExceptionGroup(
2024-10-23T01:34:19.293361078Z   | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
2024-10-23T01:34:19.293365205Z   +-+---------------- 1 ----------------
2024-10-23T01:34:19.293369054Z     | Traceback (most recent call last):
2024-10-23T01:34:19.293372827Z     |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 253, in wrap
2024-10-23T01:34:19.293377407Z     |     await func()
2024-10-23T01:34:19.293381185Z     |   File "/home/user/.local/lib/python3.11/site-packages/starlette/responses.py", line 242, in stream_response
2024-10-23T01:34:19.293385695Z     |     async for chunk in self.body_iterator:
2024-10-23T01:34:19.293389600Z     |   File "/home/user/comps/agent/langchain/src/strategy/react/planner.py", line 59, in stream_generator
2024-10-23T01:34:19.293394292Z     |     async for chunk in self.app.astream(initial_state, config=config):
2024-10-23T01:34:19.293398495Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain/agents/agent.py", line 1810, in astream
2024-10-23T01:34:19.293402938Z     |     async for step in iterator:
2024-10-23T01:34:19.293406834Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain/agents/agent_iterator.py", line 266, in __aiter__
2024-10-23T01:34:19.293411335Z     |     async for chunk in self.agent_executor._aiter_next_step(
2024-10-23T01:34:19.293415440Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain/agents/agent.py", line 1556, in _aiter_next_step
2024-10-23T01:34:19.293419847Z     |     result = await asyncio.gather(
2024-10-23T01:34:19.293423638Z     |              ^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293427425Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain/agents/agent.py", line 1589, in _aperform_agent_action
2024-10-23T01:34:19.293431945Z     |     observation = await tool.arun(
2024-10-23T01:34:19.293435820Z     |                   ^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293439807Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_core/tools/base.py", line 808, in arun
2024-10-23T01:34:19.293444185Z     |     raise error_to_raise
2024-10-23T01:34:19.293448095Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_core/tools/base.py", line 775, in arun
2024-10-23T01:34:19.293457149Z     |     response = await asyncio.create_task(coro, context=context)  # type: ignore
2024-10-23T01:34:19.293462938Z     |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293467000Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_core/tools/base.py", line 575, in _arun
2024-10-23T01:34:19.293471658Z     |     return await run_in_executor(None, self._run, *args, **kwargs)
2024-10-23T01:34:19.293475652Z     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293479627Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 590, in run_in_executor
2024-10-23T01:34:19.293484098Z     |     return await asyncio.get_running_loop().run_in_executor(
2024-10-23T01:34:19.293488127Z     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293492095Z     |   File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
2024-10-23T01:34:19.293496458Z     |     result = self.fn(*self.args, **self.kwargs)
2024-10-23T01:34:19.293500625Z     |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293504647Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_core/runnables/config.py", line 581, in wrapper
2024-10-23T01:34:19.293509081Z     |     return func(*args, **kwargs)
2024-10-23T01:34:19.293512987Z     |            ^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293516940Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_community/tools/ddg_search/tool.py", line 73, in _run
2024-10-23T01:34:19.293521349Z     |     return self.api_wrapper.run(query)
2024-10-23T01:34:19.293525125Z     |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293529015Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_community/utilities/duckduckgo_search.py", line 97, in run
2024-10-23T01:34:19.293533674Z     |     results = self._ddgs_text(query)
2024-10-23T01:34:19.293537645Z     |               ^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293541618Z     |   File "/home/user/.local/lib/python3.11/site-packages/langchain_community/utilities/duckduckgo_search.py", line 64, in _ddgs_text
2024-10-23T01:34:19.293546918Z     |     ddgs_gen = ddgs.text(
2024-10-23T01:34:19.293551265Z     |                ^^^^^^^^^^
2024-10-23T01:34:19.293555065Z     |   File "/home/user/.local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py", line 239, in text
2024-10-23T01:34:19.293559465Z     |     results = self._text_api(keywords, region, safesearch, timelimit, max_results)
2024-10-23T01:34:19.293563569Z     |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293567909Z     |   File "/home/user/.local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py", line 273, in _text_api
2024-10-23T01:34:19.293577535Z     |     vqd = self._get_vqd(keywords)
2024-10-23T01:34:19.293581575Z     |           ^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293585360Z     |   File "/home/user/.local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py", line 134, in _get_vqd
2024-10-23T01:34:19.293590235Z     |     resp_content = self._get_url("POST", "https://duckduckgo.com", data={"q": keywords})
2024-10-23T01:34:19.293595707Z     |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2024-10-23T01:34:19.293599925Z     |   File "/home/user/.local/lib/python3.11/site-packages/duckduckgo_search/duckduckgo_search.py", line 129, in _get_url
2024-10-23T01:34:19.293604360Z     |     raise RatelimitException(f"{resp.url} {resp.status_code} Ratelimit")
2024-10-23T01:34:19.293608938Z     | duckduckgo_search.exceptions.RatelimitException: https://duckduckgo.com/ 202 Ratelimit
2024-10-23T01:34:19.293613247Z     +------------------------------------