Closed statefb closed 3 months ago
example usage:
import { bedrock } from "@cdklabs/generative-ai-cdk-constructs";
import {
CharacterFilterType,
TokenFilterType,
TokenizerType,
} from "@cdklabs/generative-ai-cdk-constructs/lib/cdk-lib/opensearchserverless";
const kb = new bedrock.KnowledgeBase(this, "KB", {
embeddingsModel: bedrock.BedrockFoundationModel.TITAN_EMBED_TEXT_V1,
instruction:
"Use this knowledge base to answer questions. Please quote the reference to explain your answers.",
analyzer: {
characterFilters: [CharacterFilterType.ICU_NORMALIZER],
tokenizer: TokenizerType.KUROMOJI_TOKENIZER,
tokenFilters: [
TokenFilterType.KUROMOJI_BASEFORM,
TokenFilterType.JA_STOP,
],
}
)
Also, could you please update the code snippet here: https://github.com/awslabs/generative-ai-cdk-constructs/blob/main/src/cdk-lib/opensearch-vectorindex/README.md#vector-index with the new analyzer props
Build is not passing: https://github.com/awslabs/generative-ai-cdk-constructs/actions/runs/9752808672/job/26916988890?pr=537
fixed
Great thank you for your contribution @statefb ! I approved, once a second reviewer will approve we will merge
close #536
Add analyzers support for the construct: https://opensearch.org/docs/latest/analyzers/
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the project license.