This pull request introduces new features to enhance the usage tracking of LLM (Language Model) per user and provides an option to set a monthly consumption limit.
The following changes have been made:
Added two MongoDB collections, userLlmUsage and llmPriceRate, to store user-specific LLM usage and the current prices of LLM respectively.
Implemented utility methods to save and validate user LLM usage.
Updated the OpenAIModelSchema to include the model type (chat, completion, embedding).
Configured the initial database settings for LLM pricing and updated the readme file with detailed instructions.
Added a new environment variable DEFAULT_USER_LIMIT_USD_MONTHLY, which allows configuring a default monthly limit for each user.
This pull request introduces new features to enhance the usage tracking of LLM (Language Model) per user and provides an option to set a monthly consumption limit.
The following changes have been made:
userLlmUsage
andllmPriceRate
, to store user-specific LLM usage and the current prices of LLM respectively.DEFAULT_USER_LIMIT_USD_MONTHLY
, which allows configuring a default monthly limit for each user.