On a high level, this PR contains a set of changes that make the mongodb-chatbot-server oriented around a more generic LLM chatbot use case, not necessarily using RAG.
Rather, RAG can be part of a new injected function in the addMessageToConversation route, GenerateUserMessage. This PR includes a module with first-class support for RAG, makeRagGenerateUserPrompt().
We want to make baked-in RAG optional for the following reasons:
Sets up RAG via tool calling in a future PR
Allows package users more flexibility with how they implement RAG or other user message preprocessing.
a. For example, a user could use chain-of-thought or other advanced prompting techniques within a GenerateUserMessage before getting the final assistant response streamed to the user.
As a result of this change, I've renamed the framework "MongoDB Chatbot Framework" in the docs to better capture the fact that the framework isn't tightly coupled to RAG. This reorientation to "chatbot" also works better with the name of the repo + Github Pages site.
MongoDB Chatbot Framework
[x] Update root README to be oriented around the framework
[x] Update docs to refer to framework as "MongoDB Chatbot Framework"
GenerateUserMessage changes
[x] Refactor RAG functionality into GenerateUserPrompt
this allows for using current modules with minimal code refactoring while making RAG optional
[x] makeRagGenerateUserPrompt to perform RAG using existing modules with minimal code changes.
[x] Use new makeRagGenerateUserPrompt in the chatbot-server-mongodb-public implementation.
[x] Document changes
Other changes
Sneaking in some other changes while making the above breaking changes.
[x] Make DataStreamer provided by default
[x] Refactor maxConversationLength to be max number of user messages.
This will make things clearer when we add function calling and the number of non-user messages becomes arbitrarily large (b/c there can be multiple assistant + function messages between user messages.)
[x] Add tool calling functionality to ChatLlm
[x] Use ChatGPT API model docs-chatbot-gpt-35-turbo-0613 in the test suite. This model (0613 version) is required to use tool calling.
However, as we're not using tool calling in any app yet, we do not need to add this model to staging or production deployments.
[x] Refactor MongoDB preprocessor to work with the 0613 model. Include few shot prompting of TypeChat.
Note: I think that in the future, we should refactor the preprocessor to use a ChatGPT function instead of typechat b/c typechat is now an unnecessary dep + also seems to be minimally maintained
[x] Rename docs to "MongoDB Chatbot Framework" and associated copy updates throughout.
Notes
The following functionality is not included in this PR in order to get these changes released quickly (given Wilco want to use our OSS framework in their quest). The following changes will be additive, non-breaking changes.
Regulate when tools are called based on message flow. see if can pass args from generateUserPrompt to subsequent LLM calls. have some additional logic like passing instructions for next LLM call in the previous one.
Add ChatGPT API 0613 version to staging and prod envs
Jira: https://jira.mongodb.org/browse/EAI-182
Changes
On a high level, this PR contains a set of changes that make the
mongodb-chatbot-serveroriented around a more generic LLM chatbot use case, not necessarily using RAG. Rather, RAG can be part of a new injected function in the addMessageToConversation route,GenerateUserMessage. This PR includes a module with first-class support for RAG,makeRagGenerateUserPrompt().We want to make baked-in RAG optional for the following reasons:
GenerateUserMessagebefore getting the final assistant response streamed to the user.As a result of this change, I've renamed the framework "MongoDB Chatbot Framework" in the docs to better capture the fact that the framework isn't tightly coupled to RAG. This reorientation to "chatbot" also works better with the name of the repo + Github Pages site.
MongoDB Chatbot Framework
GenerateUserMessagechangesGenerateUserPromptmakeRagGenerateUserPromptto perform RAG using existing modules with minimal code changes.makeRagGenerateUserPromptin thechatbot-server-mongodb-publicimplementation.Other changes
Sneaking in some other changes while making the above breaking changes.
DataStreamerprovided by defaultdocs-chatbot-gpt-35-turbo-0613in the test suite. This model (0613 version) is required to use tool calling.0613 model. Include few shot prompting of TypeChat.Notes
The following functionality is not included in this PR in order to get these changes released quickly (given Wilco want to use our OSS framework in their quest). The following changes will be additive, non-breaking changes.
These changes will be added as part of EAI-121