boto / boto3

AWS SDK for Python
https://aws.amazon.com/sdk-for-python/
Apache License 2.0
9.08k stars 1.87k forks source link

`bedrock-runtime` client `apply_guardrail` function adds new line at the end of masked text #4317

Open MottiG opened 1 month ago

MottiG commented 1 month ago

Describe the bug

When using bedrock-runtime apply_guardrail on a given text, where guardrail is configured with mask action, the returned text is indeed masked where needed, but a new line is added to the given input.

TEXT = "The official address of the UK Prime Minister\'s office is:\\n\\n10 Downing Street\\nLondon\\nSW1A 2AA\\nUnited Kingdom\\n\\nThis address is commonly referred to simply as \\"10 Downing Street\\" or \\"Number 10.\\" It has been the official residence and office of the British Prime Minister since 1735. The building is located in central London, near the Houses of Parliament and Whitehall."

bedrock = boto3.client(...).   # bedrock has a configured guardrails with "ADDRESS" PII and action "mask"
guard_res = bedrock.apply_guardrail(
    guardrailIdentifier=...,
    guardrailVersion=..., 
    source="OUTPUT",
    content=[{'text':{'text':TEXT}}]
)

Regression Issue

Expected Behavior

Bedrock should return the text as is, while masking the configured addresses without a newline at the end.

guard_res['outputs'][0]['text'] should be:

"The official address of the UK Prime Minister\'s office is:\n\n{ADDRESS}\n{ADDRESS}\n\nThis address is commonly referred to simply as "{ADDRESS}." It has been the official residence and office of the British Prime Minister since 1735. The building is located in central {ADDRESS}, near the Houses of Parliament and Whitehall."

Current Behavior

Actual results (guard_res['outputs'][0]['text']):

"The official address of the UK Prime Minister\'s office is:\n\n{ADDRESS}\n{ADDRESS}\n\nThis address is commonly referred to simply as "{ADDRESS}." It has been the official residence and office of the British Prime Minister since 1735. The building is located in central {ADDRESS}, near the Houses of Parliament and Whitehall.\n"

As you can see when scrolling to the end of the resulting text, a newline character \n was added to the end of the text.

Reproduction Steps

TEXT = "The official address of the UK Prime Minister\'s office is:\\n\\n10 Downing Street\\nLondon\\nSW1A 2AA\\nUnited Kingdom\\n\\nThis address is commonly referred to simply as \\"10 Downing Street\\" or \\"Number 10.\\" It has been the official residence and office of the British Prime Minister since 1735. The building is located in central London, near the Houses of Parliament and Whitehall."

bedrock = boto3.client(...).   # bedrock has a configured guardrails with "ADDRESS" PII and action "mask"
guard_res = bedrock.apply_guardrail(
    guardrailIdentifier=...,  # add guardrail id
    guardrailVersion=...,  # add guardrail version or 'DRAFT'
    source="OUTPUT",
    content=[{'text':{'text':TEXT}}]
)

print(guard_res['outputs'][0]['text'].endswith('\n') is True)
print(guard_res['outputs'][0]['text'])

Possible Solution

I'm unsure of the root cause of this behavior, but I see that the client already got the new line char as a response to the API call.

Additional Information/Context

I also checked the usage of configuredGuardrail in the call for the converse function. The behavior didn't reproduce, and the guarded text returned as expected.

SDK version used

1.34.162

Environment details (OS name and version, etc.)

MacOS 14.7

tim-finnigan commented 4 weeks ago

Thanks for reaching out. I could reproduce the behavior you described where \n was added to the text output when applying a guardrail with "Address" PII and action "mask". The apply_guardrail command makes a call to the underlying ApplyGuardrail API, so this is an issue with the Bedrock service API rather than with boto3 directly.

If you add boto3.set_stream_logger('') to your script then it will show the full debug logs and what the response looks like from the Bedrock service API. When doing that, I see the \n is in the API response body, which confirms that this is an issue with the API rather than the SDK. I'll go ahead and forward this issue to the Bedrock team internally for further review, and will share any updates here.