search

aws-solutions / generative-ai-application-builder-on-aws

Generative AI Application Builder on AWS facilitates the development, rapid experimentation, and deployment of generative artificial intelligence (AI) applications without requiring deep experience in AI. The solution includes integrations with Amazon Bedrock and its included LLMs, such as Amazon Titan, and pre-built connectors for 3rd-party LLMs.
https://aws.amazon.com/solutions/implementations/generative-ai-application-builder-on-aws
Apache License 2.0
161 stars 55 forks source link

When updating a use case, setting the Score threshold to DISABLED does not seem to take effect. #154

Open hiraken-w opened 1 week ago

hiraken-w commented 1 week ago

Describe the bug

As per the following documentation, we can configure Kendra's Advanced RAG configurations. However, once we set the Score threshold to a value other than DISABLED, like LOW, we cannot revert it back to DISABLED in subsequent updates. On the web app, there is no error shown, and it appears to be successful, but the change is not reflected.

https://docs.aws.amazon.com/solutions/latest/generative-ai-application-builder-on-aws/advanced-knowledge-base-settings.html

If we update to set the Score threshold to DISABLED, the request body for the PATCH method to API Gateway does not include the ScoreThreshold parameter. In this case, it is considered necessary to perform processing such as deleting the item keys (ScoreThreshold, StorageCapacityUnits) from the DynamoDB table.

To Reproduce

  1. Create a Generative AI Application Builder.
  2. Deploy a new use case. Leave the default settings as is. Select Kendra for the Knowledge base type, and set a value other than DISABLED for the Score Threshold in Advanced RAG configurations.
  3. Update the use case. Set DISABLED for the Score Threshold in Advanced RAG configurations.

As a result, the Score Threshold does not become DISABLED.

Expected behavior

The Score Threshold value in Advanced RAG configurations can be updated to DISABLED.

Please complete the following information about the solution:

ihmaws commented 1 week ago

Thank you @hiraken-w for the detailed bug report!

I was able to reproduce the issue and identify the root cause. The team will have a look at a fix and include a patch in our next release.

We'll let you know here once a fix is available. Thanks again!