search

aws-samples / aws-agentic-document-assistant

An agent based LLM assistant that extends RAG with batch entity extraction and SQL querying to improve performance on multi-step and analytical questions.
MIT No Attribution
45 stars 19 forks source link

Debug steps / suggestions ? #9

Open awsandy opened 1 month ago

awsandy commented 1 month ago

Hi,

I have deployed this successfully - and run the 04- notebook to load the demo data. Both LoadEntitiesToSQLTable & EmbeddingCreationAndIndexing report "succeeded"

The Amplify CI/CD build worked ok and I can register login ok.

As far as I can tell everything has worked ok.

But when I enter the question

Give me the top 5 companies with the highest revenue in the last 2 years and identify the risks associated with the top one

I get this unexpected response:

_Unfortunately I do not have enough context to provide the specific company revenue information you requested. However, I can offer some general thoughts on identifying risks associated with top revenue companies:

The company with the highest revenue likely has significant market share in their industry, which can come with risks like antitrust scrutiny or disruption from new innovations. Being reliant on a single or few high-revenue products/services is also a risk if consumer demand changes.

Other potential risks include overexpansion, taking on too much debt, compliance issues as the company scales, cybersecurity threats with large databases of customer data, and challenges retaining talent. Leadership problems or scandals would additionally pose reputation risk.

Geopolitical factors, trade policy, and regional instability could uniquely impact multinational corporations with the highest revenues. It's important to assess risks holistically across operations, finances, company culture, industry trends, and macroeconomic conditions. Let me know if you would like me to elaborate on any particular aspect of risk analysis for top revenue companies._

The Lambda AssistantBackendStack-LambdaAgentContainer* cloudwatch log ends with:

2024-05-10T17:50:33.297ZThe current user input is the following: Give me the top 5 companies with the highest revenue in the last 2 years and identify the risks associated with the top one 2024-05-10T17:50:33.297Z Assistant: Assistant: 2024-05-10T17:50:40.817Z Finished chain. 2024-05-10T17:50:40.819ZEND RequestId: 417702b4-e045-4fa3-a18b-5e309c56396f

Can you suggest some things to check / debug steps please?

awsandy commented 1 month ago

Ah - the streamlit app has an option to pass "agentic" mode to the backend Lambda - but the Amplify app doesn't seem to have this - unless I've missed something ?