aws-samples / amazon-bedrock-workshop

This is a workshop designed for Amazon Bedrock a foundational model service.
https://catalog.us-east-1.prod.workshops.aws/workshops/a4bdb007-5600-4368-81c5-ff5b4154f518/en-US/20-intro
MIT No Attribution
1.34k stars 576 forks source link

aws-bedrock agent failing infrequently with failureReason': 'LLM prediction format incorrect' #192

Closed abhitjet closed 6 months ago

abhitjet commented 6 months ago

Hi, I have created an agent and it always works fine when tested with aws console. Agent contains a lambda function which calls claude for some generation activity. It also has a matching swagger to trigger operations of function. However, when trigger it(agent) from code(lambda function with python), providing agent and alias id, it works as expected sometimes and fails quite a lot of times with below error -

trace': {'agentId': 'SJ4AHHVXX', 'agentAliasId': 'SZD0OGXXX', 'sessionId': 'SESSMAINT1', 'trace': {'failureTrace': {'traceId': 'd34fe4ba-78b6-4ccf-8aa5-682e74890d31-0', 'failureReason': 'LLM prediction format incorrect'}}}} {'chunk': {'bytes': b'Sorry, I am unable to assist you with this request.'}}

{'chunk': {'bytes': b'Sorry, I am unable to assist you with this request.'}}

sample lambda code -

def invoke_maintenance_agent(searchterm): try: response = bedrock.invoke_agent( agentId='SJ4AHHVXX', agentAliasId='SZD0OGXXX', sessionId='SESSMAINT1', inputText=searchterm ) completion = response.get('completion') if completion: print('inside completion') all_records=[] for event in completion: # loop over to get each record print(event) if 'chunk' in event: chunk = event.get('chunk') bytes = chunk.get('bytes') all_records.append(bytes) agentResponse = str(all_records[0],'utf-8') else: agentResponse = 'Some issue in invoking agent' return agentResponse

except Exception as e: 
     raise RuntimeError(f"Exception - {str(e)}") 

Kindly assist. Thank You.

abhitjet commented 6 months ago

I was able to solve this error same evening I posted the issue. Guess I should have posted here earlier. It actually meant that I was not able to trigger the underlying lambda in the agent. Reason, I suppose is due to some session mgmt issues. Managing the session id fixed this issue.