Slowness with latest update and brain agent required tool name with question

[satishthumar]

We’ve noticed that the latest updates have resulted in a decrease in performance speed. Our bot, which was developed using this code in conjunction with two different tools, employs GPT-4 with a capacity of 100 PTUs. Despite this, we’re experiencing response times of 30-40 seconds for both RAG search and SQL queries. Furthermore, we’ve detected that the absence of specific tool parameters leads to inadequate responses, an issue that wasn’t present before the recent updates.”

[pablomarin]

Can you be more specific regarding; "absence of specific tool parameters "? are you refering to return_direct=True?

[satishthumar]

In prompt we have to specify (i.e. doc or sql) tool name prior to question. Agent is not learning on its own to direct question based on tools knowleadge to a particular tool.

We are using apps/backend/botservice and it does not have any settings set to use return_direct=true. We are using bot.py and utils.py.

    doc_search = DocSearchAgent(llm=llm, indexes=doc_indexes,
                       k=6, reranker_th=1,
                       sas_token=os.environ['BLOB_SAS_TOKEN'],
                       name="doc",
                       description="useful when the questions includes the term: doc",
                       callback_manager=cb_manager, verbose=False,streaming=True)

    sql_search = SQLSearchAgent(llm=llm, k=30, callback_manager=cb_manager,
                        name="sql",
                        description="useful when the questions includes the term: sql",
                        verbose=False,streaming=True)

    tools = [doc_search, sql_search]

Let me know if you need any more information. Also we are unable to get streaming response although we set streaming=true.

Thank you for your quick checking.

[pablomarin]

You tell the brain agent when to use each tool based on the description of the tool, so change the description to what you want : “useful when …” Regarding steaming, bot service doesn’t support streaming , that’s why we added langserve as an option backend

On Tue, Apr 23, 2024 at 9:36 AM satishthumar @.***> wrote:

In prompt we have to specify (i.e. doc or sql) tool name prior to question. Agent is not learning on its own to direct question based on tools knowleadge to a particular tool.

We are using apps/backend/botservice and it does not have any settings set to use return_direct=true. We are using bot.py and utils.py. doc_search = DocSearchAgent(llm=llm, indexes=doc_indexes, k=6, reranker_th=1, sas_token=os.environ['BLOB_SAS_TOKEN'], name="doc", description="useful when the questions includes the term: doc", callback_manager=cb_manager, verbose=False,streaming=True)

sql_search = SQLSearchAgent(llm=llm, k=30, callback_manager=cb_manager,
                    name="sql",
                    description="useful when the questions includes the term: sql",
                    verbose=False,streaming=True)

tools = [doc_search, sql_search]

Let me know if you need any more information. Also we are unable to get streaming response although we set streaming=true.

Thank you for you quick checking.

— Reply to this email directly, view it on GitHub https://github.com/MSUSAzureAccelerators/Azure-Cognitive-Search-Azure-OpenAI-Accelerator/issues/63#issuecomment-2072499451 or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUPWIHFWAQLSWUPCJCOBELY6ZWWJBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVE3DGOJVHE3TGNJQQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRSGI2TMOJXGA4DKN5HORZGSZ3HMVZKMY3SMVQXIZI . You are receiving this email because you commented on the thread.

Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

[satishthumar]

Thank you for informaiton. We will evaluate LangServe for streaming. One more question, how to have feedback (Like,DisLike) loop for the bot response?

Any suggestion to have quick response for DocSearchAgent as it is taking 30-40 seconds with 100 PTU?