search

neokd / NeoGPT

Chat effortlessly, execute commands, and interpret code with Llama3, Phi3, and more - your local AI assistant. Enjoy seamless interaction while ensuring ultimate privacy
https://neogpt.dev
MIT License
68 stars 62 forks source link

Budget Manager #197

Closed neokd closed 3 months ago

neokd commented 4 months ago

In the budget_manager.py file, implement code within the budget_manager function to compute the budget and update the TOTAL_COST. In the chat session, if a user sets a maximum budget and surpasses it, gracefully conclude the session with a message indicating that the budget has been exceeded.

ayushmorbar commented 4 months ago

Hi, Assign me this issue as part of JWOC. Do review my current PR

ayushmorbar commented 4 months ago

Could you explain more on this issue, like what am i supposed to do exactly?

neokd commented 4 months ago

In the callback_handler.py file, there exists a Token Callback Handler responsible for calculating the generated tokens. To enhance user control, a new feature is expected with the --max-budget flag. This flag allows users to set a predefined maximum budget for a session. If the user surpasses this allocated budget, the session can either be terminated, or it can continue until the specified budget is exhausted.

Make sure you update your fork before working on it. And if any clarifications ping me here.

ayushmorbar commented 4 months ago

Understood, lemme give it a try.

neokd commented 4 months ago

Un assigning!