A modular and comprehensive solution to deploy a Multi-LLM and Multi-RAG powered chatbot (Amazon Bedrock, Anthropic, HuggingFace, OpenAI, Meta, AI21, Cohere, Mistral) using AWS CDK on AWS
Currently cross encoder models are used to rank the search results but the models available need to be hosted on Sagemaker which increases cost significantly. Having an option to disable cross encoder models would be helpful while exploring the chatbot so that Sagemaker costs can be avoided.
Changes:
Disabling the use of Sagemaker in the CLI will change the list of available models in the config. Based on the available models, cross encoders will be available or not. (since the only cross encoder models is in SageMaker, it will disable it)
Make the cross encoder parameter optional during the workspace creation
Add an integ test.
Thank you @azaylamba for your help with this change.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Issue #, if available: https://github.com/aws-samples/aws-genai-llm-chatbot/issues/222
Description of changes: This change is a follow up of https://github.com/aws-samples/aws-genai-llm-chatbot/pull/286 by @azaylamba
Currently cross encoder models are used to rank the search results but the models available need to be hosted on Sagemaker which increases cost significantly. Having an option to disable cross encoder models would be helpful while exploring the chatbot so that Sagemaker costs can be avoided.
Changes:
Thank you @azaylamba for your help with this change.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.