search

aws-samples / bedrock-access-gateway

OpenAI-Compatible RESTful APIs for Amazon Bedrock
MIT No Attribution
224 stars 46 forks source link

run non-stream requests as async calls to avoid blocking requests #23

Open eliran89c opened 3 months ago

eliran89c commented 3 months ago

*Issue #22

Description of changes: This PR demonstrates that running boto3 in async mode resolves the issue described.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

daixba commented 3 months ago

Thanks for the PR, I will need to validate it first. Will try to update as soon as I can.

daixba commented 3 months ago

For urgent usage, please do a customization to the code and deploy and host your own ECR images.

eliran89c commented 2 months ago

For urgent usage, please do a customization to the code and deploy and host your own ECR images.

Thanks, we are using custom deployment for EKS but I believe this is something others will benefit from