Feature Request: Support for Bedrock Converse

[AndreCNF]

• Async AWS SDK for Python version: 2.1646.0
• Python version: 3.10.13
• Operating System: MacOS

Description

I'm trying to use the bedrock-runtime service with the new Converse API. However, aioboto3's client doesn't find the converse method.

This might just need a bump in the boto3 and/or botocore dependencies, as was the case in #317.

What I Did

Tried implementing async methods in LlamaIndex's BedrockConverse LLM class, but fails because the async client' bedrock-runtime service doesn't have the converse method.

[terricain]

You'll need a bump of aiobotocore first, lemme know once that's done and i'll update.

[AndreCNF]

You'll need a bump of aiobotocore first, lemme know once that's done and i'll update.

I've opened an issue in aiobotocore ✅

[jscottcronin]

👀 Following as I'd like to use bedrock runtime client to access LLM models via the .invoke_model method

[AndreCNF]

You'll need a bump of aiobotocore first, lemme know once that's done and i'll update.

I've opened an issue in aiobotocore ✅

aiobotocore has been updated ✅ @terricain Can you update aioboto3 then?

[terricain]

v13.1.0 is out, try that!

[AndreCNF]

v13.1.0 is out, try that!

I'm still hitting this error on version v13.1.0:

session_kwargs = {
    "profile_name": profile_name,
    "region_name": region_name,
    "aws_access_key_id": aws_access_key_id,
    "aws_secret_access_key": aws_secret_access_key,
    "aws_session_token": aws_session_token,
    "botocore_session": botocore_session,
}
asession = aioboto3.Session(**session_kwargs)
aclient = asession.client("bedrock-runtime", config=config)
...
await aclient.converse(**kwargs)

AttributeError: 'ClientCreatorContext' object has no attribute 'converse'.

Any idea why I'm still not getting the Converse API in the aioboto3 client?

[terricain]

import asyncio
import aioboto3

async def main():
    session = aioboto3.Session(region_name='eu-west-2')
    async with session.client("bedrock-runtime") as client:
        messages = [{"role": "user", "content": [{"text": "What is your name?"}]}]

        response = await client.converse(
            modelId="amazon.titan-text-lite-v1",
            messages=messages,
        )

        print(response['output'])

asyncio.run(main())

This works with v13.1.0 in a fresh virtualenv. I can only imagine that you've pinned boto3/botocore to an older version.

[AndreCNF]

import asyncio
import aioboto3

async def main():
    session = aioboto3.Session(region_name='eu-west-2')
    async with session.client("bedrock-runtime") as client:
        messages = [{"role": "user", "content": [{"text": "What is your name?"}]}]

        response = await client.converse(
            modelId="amazon.titan-text-lite-v1",
            messages=messages,
        )

        print(response['output'])

asyncio.run(main())

This works with v13.1.0 in a fresh virtualenv. I can only imagine that you've pinned boto3/botocore to an older version.

Thanks for your comment! My issue seems to have been that I was instantiating the client without the async with. Now it works ✅