search

aws-samples / bedrock-access-gateway

OpenAI-Compatible RESTful APIs for Amazon Bedrock
MIT No Attribution
124 stars 24 forks source link

when the parameter 'messages' length exceeds 144, there will be no response. #17

Open dakai6677 opened 1 month ago

dakai6677 commented 1 month ago

the input token count is around 16000 model is 'anthropic.claude-3-sonnet-20240229-v1:0'

curl http://Bedroc-Proxy-xxxx.us-east-1.elb.amazonaws.com/api/v1/chat/completions
-H "Content-Type: application/json"
-H "Authorization: Bearer xxxx"
-d @/home/ec2-user/param.json

daixba commented 1 month ago

This may be a limit of Amazon Bedrock. Can you please:

  1. Try to use the SDK directly and see if the issue exists: https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-anthropic-claude-messages.html
  2. Provide the detailed response (like the finish reason etc.), and if possible, please share the input (if there is no sensitive data)
  3. See if you can limit the number of messages (chat history) from you side.