[Amazon Bedrock]: Amazon Bedrock Guardrail Contextual grounding check is not mapped.

Integration Name

Amazon Bedrock [aws_bedrock]

Dataset Name

logs-aws_bedrock.invocation

Integration Version

0.11.3

Agent Version

8.15.3

Agent Output Type

elasticsearch

Elasticsearch Version

8.15.3

OS Version and Architecture

Ubuntu

Software/API Version

Error Message

No error message presented.

Event Original

{"schemaType":"ModelInvocationLog","schemaVersion":"1.0","timestamp":"2024-11-21T11:48:13Z","accountId":"891377031307","identity":{"arn":"arn:aws:iam::891377031307:user/shashank"},"region":"us-east-1","requestId":"787423d4-0618-4465-8835-10161b3ebc12","operation":"Converse","modelId":"anthropic.claude-3-5-sonnet-20240620-v1:0","input":{"inputContentType":"application/json","inputBodyJson":{"messages":[{"role":"user","content":[{"guardContent":{"text":{"text":"The moon is composed primarily of rock and minerals, and there is no evidence of cheese or mining activities there.","qualifiers":["grounding_source"]}}},{"guardContent":{"text":{"text":"I read that the moon is made entirely of cheese and that astronauts have been mining it for years to supply dairy products on Earth. Can you explain how they process the cheese and distribute it globally?","qualifiers":["query","guard_content"]}}}]}]},"inputTokenCount":70},"output":{"outputContentType":"application/json","outputBodyJson":{"output":{"message":{"role":"assistant","content":[{"text":"GuardRail Blocked : The model cannot answer this question"}]}},"stopReason":"guardrail_intervened","metrics":{"latencyMs":5301},"usage":{"inputTokens":70,"outputTokens":209,"totalTokens":279},"trace":{"guardrail":{"modelOutput":["{\"id\":\"msg_bdrk_01BHJB9SrVhmZv6Ktq7MwLru\",\"type\":\"message\",\"role\":\"assistant\",\"model\":\"claude-3-5-sonnet-20240620\",\"content\":[{\"type\":\"text\",\"text\":\"I apologize, but I need to correct some misinformation in your statement. The moon is not made of cheese, and astronauts are not mining it for dairy products. This is a common myth or joke, but it's not based in reality. Here are the facts:\\n\\n1. The moon is indeed composed primarily of rock and minerals, as you correctly stated at the beginning of your message.\\n\\n2. There is no evidence of cheese or any organic matter on the moon.\\n\\n3. Astronauts have visited the moon for scientific exploration, not for mining or harvesting any resources.\\n\\n4. The idea of the moon being made of cheese is a popular cultural reference and children's tale, but it's not factual.\\n\\n5. Any dairy products we consume on Earth are produced here on our planet, not from the moon.\\n\\nIf you're interested in learning more about the actual composition and exploration of the moon, I'd be happy to provide factual information on those topics.\"}],\"stop_reason\":\"end_turn\",\"stop_sequence\":null,\"usage\":{\"input_tokens\":70,\"output_tokens\":209}}"],"inputAssessment":{"iy5aqw3mmiwu":{"invocationMetrics":{"guardrailProcessingLatency":302,"usage":{"topicPolicyUnits":1,"contentPolicyUnits":1,"wordPolicyUnits":1,"sensitiveInformationPolicyUnits":1,"sensitiveInformationPolicyFreeUnits":0,"contextualGroundingPolicyUnits":0},"guardrailCoverage":{"textCharacters":{"guarded":204,"total":321}}}}},"outputAssessments":[{"iy5aqw3mmiwu":{"contextualGroundingPolicy":{"filters":[{"type":"GROUNDING","threshold":0.99,"score":0.36,"action":"BLOCKED"},{"type":"RELEVANCE","threshold":0.99,"score":0.97,"action":"BLOCKED"}]},"invocationMetrics":{"guardrailProcessingLatency":385,"usage":{"topicPolicyUnits":1,"contentPolicyUnits":1,"wordPolicyUnits":1,"sensitiveInformationPolicyUnits":1,"sensitiveInformationPolicyFreeUnits":0,"contextualGroundingPolicyUnits":2},"guardrailCoverage":{"textCharacters":{"guarded":915,"total":915}}}}}]}}},"outputTokenCount":209}}

What did you do?

Executed a Prompt with words that can fail grounding check in AWS bedrock models. Example of the Prompt

"I read that the moon is made entirely of cheese and that astronauts have been mining it for years to supply dairy products on Earth. Can you explain how they process the cheese and distribute it globally?"

Reference Source for Grounding Supplied

The moon is composed primarily of rock and minerals, and there is no evidence of cheese or mining activities there.

What did you see?

Although the event.original presents with a BLOCKED action based on contextual grounding checks, no mapping to query the same via fileds.

Sample Snippet from event.original

"outputAssessments":[{"iy5aqw3mmiwu":{"contextualGroundingPolicy":{"filters":[{"type":"GROUNDING","threshold":0.99,"score":0.36,"action":"BLOCKED"},{"type":"RELEVANCE","threshold":0.99,"score":0.97,"action":"BLOCKED"}]},"invocationMetrics":{"guardrailProcessingLatency":385,"usage":{"topicPolicyUnits":1,"contentPolicyUnits":1,"wordPolicyUnits":1,"sensitiveInformationPolicyUnits":1,"sensitiveInformationPolicyFreeUnits":0,"contextualGroundingPolicyUnits":2},"guardrailCoverage":{"textCharacters":{"guarded":915,"total":915}}}}}]}}},"outputTokenCount":209}}

All policy related fields are not populated , including the basic gen_ai.policy.action : BLOCKED

What did you expect to see?

Filed mappings for GROUNDING and RELEVANCE with the respective threshold and action

Anything else?

Identified when working on use cases for https://github.com/elastic/security-team/issues/11054

elastic / integrations