How to pass BEDROCK_ASSUME_ROLE to BedrockChat and use the default credential chain?

[vikramelango]

Your Question what is unclear to you? What would you like to know?

According to the ragas Bedrock documentation you show how to pass credentials_profile_name to instantiate BedrockChat Model. I need to pass BEDROCK_ASSUME_ROLE instead, is there away to pass this IAM role that has permissions to access Bedrock?

Code Examples This community speaks code. Share your code snippets to help us understand your question better.

Additional context Anything else you want to share with us?

[shahules786]

@arm-diaz @austinmw Can you guys help?

[vikramelango]

I found a way around this, anyone who is trying the same approach might benefit from this. I created a client using assumed_role and region, created LLM langchain object with the client. Passed the llm object directly to ragas evaluate function to make this work with out the credentials_profile_name.

1. get_bedrock_client is same as standard way to create bedrock client. Refer notebook

def get_client():

    boto3_bedrock = get_bedrock_client(
        assumed_role=os.environ.get("BEDROCK_ASSUME_ROLE", None),
        region=os.environ.get("AWS_DEFAULT_REGION", None)
    )
    return boto3_bedrock

2. create llm Bedrock object using langchain

from langchain.llms.bedrock import Bedrock

llm = Bedrock(
    client=get_client(),
    model_id="anthropic.claude-v2",
    model_kwargs={"max_tokens_to_sample": 500, "temperature": 0.9}
    )

3. Pass llm to ragas evaluate

from ragas import evaluate
import nest_asyncio  # CHECK NOTES

# NOTES: Only used when running on a jupyter notebook, otherwise comment or remove this function.
nest_asyncio.apply()

result = evaluate(
    amnesty_qa["eval"].select(range(3)),
    metrics=metrics,
    llm=llm,
    embeddings=bedrock_embeddings,
)

result

One note, I was running into to issues with evaluate function streaming for async operation when using this approach. I will open an separate issue with details.

[subha-aws]

from langchain.llms.bedrock import Bedrock
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
from ragas.llms import LangchainLLMWrapper
from ragas.embeddings.base import LangchainEmbeddingsWrapper

boto3_bedrock=get_client()
bedrock_llm = Bedrock(
    model_id="cohere.command-text-v14",
    client=boto3_bedrock,
    model_kwargs={"temperature": 0.5, "max_tokens": 1000},
    streaming=True,
    callbacks=[StreamingStdOutCallbackHandler()],
)

ragas_critic_llm = LangchainLLMWrapper(bedrock_llm)

bedrock_embeddings = BedrockEmbeddings(
    model_id="amazon.titan-embed-text-v1", client=boto3_bedrock
)
ragas_embeddings=LangchainEmbeddingsWrapper(bedrock_embeddings)

result = evaluate(
    amnesty_qa["eval"].select(range(3)),
    metrics=metrics,
    llm=ragas_critic_llm ,
    embeddings=ragas_embeddings,
)

Try setting streaming=True and set a callback handler. Also we have to wrap the LLMs in LangchainLLMWrapper embeddings model should be wrapped in LangchainEmbeddingsWrapper