Daily budget cap

[MBudkin]

Is it possible to do similar to "MONTHLY_USER_BUDGETS", only "DAY_USER_BUDGETS"?

This would allow users to use the bot on a daily basis, while having a limit of tokens per day. If the user has increased the daily rate, then he will be able to use the bot the next day.

[AlexHTW]

Hey, thanks for the suggestion. I have some reservations. I think it might over complicate things for the average user and only be useful to very few. Firstly, the additional management of daily budgets per user in the .env file could become confusing. Secondly, the relationship between daily and monthly budget could be unintuitive - for example, what if a user increases their daily rate but has reached their monthly limit? You would have to increase the monthly as well. So, in my opinion it would be best to only use either monthly or daily budgets.

This option could be added to the project by refactoring MONTHLY_USER_BUDGETS to USER_BUDGETS and add a new environment variable that allows to set if the USER_BUDGETS are applied DAILY or MONTHLY.

@MBudkin would that option allow you to accomplish your goals or would you really need to have both budgets in parallel?

[MBudkin]

@AlexHTW You mean for each user it would be possible to configure individually - that would be great! And also so that the server could specify what will be used for everyone by default: monthly or daily. So that you can set up on your server an accounting for the daily amount of funds and specify a price for everyone. And for individual users, you could specify other parameters.

[AlexHTW]

Sorry, no I meant only globally if the user budgets are taken as monthly or daily (I sometimes used user as users of the users of the repo and sometimes as users of your bot, my bad). But I understand your use case better now: the advantage of having monthly and daily budgets would be that some users could be limited daily and other users monthly. In that case it's probably really more straight forward to have MONTHLY_USER_BUDGETS and DAILY_USER_BUDGETS and maybe a designated symbol (for example "*") to assign no limitations to that user.

For example:

ALLOWED_TELEGRAM_USER_IDS="USER_1,USER_2,USER_3,USER_4"
MONTHLY_USER_BUDGETS ="*,100.0,*,100.0"
DAILY_USER_BUDGETS ="*,*,10.0, 10.0"

Would mean: USER_1 has unlimited access USER_2 has a monthly budget of 100 and can use it all in one day. USER_3 has a daily budget of 10 and no monthly limitations (so could use up to $310 a month) USER_4 has a daily budget of 10 and a monthly budget of 100 and has to stay within both.

Edit: setting MONTHLY_USER_BUDGETS=* or DAILY_USER_BUDGETS=* would remove monthly or daily restrictions for all users.

What do you think @MBudkin , @n3d1117 ?

[MBudkin]

@AlexHTW Great idea! It would be convenient if users and their parameters could be specified in a separate file, like this:

User ID, name, daily limit, monthly limit

11223344,Max,*,*
12345678,John,5.0,70.0
87654321,Anna,10.0,100.0
44332211,Jack,*,80.0
88776655,my best friend Harry,15.0,*

The name is used only for human understanding. You can take it automatically from telegrams, or manually enter it, it does not affect anything. Instead of a name, you can write anything, for example, comments for yourself.

[AlexHTW]

@MBudkin Your suggestion would really improve the readability and maintanability of the budget topic. On the other hand I think there are benefits to having all the settings in one place. Especially for people setting it up directly with the docker image. But I can't really say much on the topic. Implementing more interactive user setups (like you suggested in #108) would be good enough for me.

I'll send a PR with the daily budget stuff in the .env when I have some time, moving it to a different file could be added later. I have a feeling it would need some more restructuring in many places.

Would you also want a DAILY_GUEST_BUDGET?

[MBudkin]

@AlexHTW Yes, DAILY_GUEST_BUDGET would be useful, because the user may suddenly use up the monthly limit and not be able to use the bot until the end of the month. And with a daily limit, the user can use the bot the very next day. Thank you for your contribution to the project!

[AlexHTW]

Hey @MBudkin, I gave the issue some more thought and would love to hear your opinion. I feel that additionally to daily budgets and monthly budgets it would also be very useful to have budgets without time restrictions. So that a user that acquired a budget amount doesn't need to spend it all in a day or month and it doesn't reset. Basically for monetization purposes - daily/monthly enables subscription based models and the unrestricted budget enables one time purchases. However, I still want to keep it relatively simple in the .env and I think that the plan I suggested before will get too complex if another list for unrestricted budgets is added. Thus, I would prefer to not mix all different budget variations in one bot and have a general setting per bot. In the .env the whole budget topic could look like this:

# select one of the following types to be applied to user and guest budgets: "daily", "monthly" or "unrestricted"
BUDGET_TYPE = "monthly" 
USER_BUDGETS = "100,100,100"
GUEST_BUDGET=20

So you would not be able to set different kinds to different users but between the three types I feel like one of those would be able to fit the needs of most bots.

[MBudkin]

Hello @AlexHTW ! I completely agree with you! The administrator can choose once which monetization model he wants to use and stick to. Separating monetization models for each user will certainly be difficult to implement in code. I believe that a one-time definition of the model will be sufficient and quite convenient. There is a choice and this is the main thing;) Thank you and good luck!

[AlexHTW]

@MBudkin awesome, thanks for your feedback! I will keep you updated on the progress :)

[Geczy]

came her efor the sam ething, definitely need a monthly global budget, not per user

[AlexHTW]

Hey @Geczy, do you mean global budget as in if the total cost of requests from all users per month? Currently we would need to read all usage_log files for every request to check that. I fear it might slow performance too much for something that is not that important to many users I suggest utilizing the soft and hard limits of the openAI billing settings for that and otherwise don't give out more budget to users than you are willing to spend in total.

[Geczy]

isnt it just a simple global counter? that ticks up with every request? should just be an integer for tokens no?

[AlexHTW]

It has to be persisted between restarts and account for the prices of the different request types.

Can you share a scenario where you need a global budget that can not be realized through user specific daily/monthly/unrestriced budgets or OpenAI soft/hard limits?

Another concern I have with a global budget is that it would lead to users hitting budget restrictions while still having personal budget left. I feel like that would nullify user specific budgets.

All in all I think adding global budget is a zero-sum game in maintenance overhead between admins who want it and those who don't. And personally I think most don't because the goal of budgets is that users can be accountable for their spending.

[Geczy]

persistence and request price types seems like two simple issues to solve. even writing a to json file would be enough. docker has built in volume persistence you can point it to the json in the repo for. the prices are in a table on openai website so that's just a simple local array

the scenario is if i want the bot to be able to join several unknown-to-me group chats, i have no idea what to set the limit to. there could be 100 users or 1000 users in those group chats. per person would mean $1 * 1000 = $1k a month? that doesn't sound logical to me

but a global $20 use it until its gone is what would work for me

as an aside, a limit per group chat would also be nice

[AlexHTW]

Thanks for your elaboration. You are right, the issues are not hard to solve and are already implemented in a similar fashion for the user tracking, my concerns are more about how these additions would interact with the existing implementations. But maybe adding a global usage tracking file would be of interest to more admins.

To prevent any conflicts with per user budgets the global budgets could be added as another type, so you either user per user daily/monthly/unrestricted budgets or a shared global budget. Would this be useful to you?

As for your example I think it can be perfectly solved with the current MONTHLY_GUEST_BUDGET. Any person that is not in the ALLOWED_TELEGRAM_USER_IDS list (meaning it should not be set to *) can use the bot in a group that has an allowed user as member, within the shared MONTHLY_GUEST_BUDGET. (note to myself or @n3d1117 if you get to it before I do , also allow group chats where a member is in the admins list)

So, if your .env looks like this:

ALLOWED_TELEGRAM_USER_IDS="_YOUR_ID_"
MONTHLY_GUEST_BUDGET=20.0

All the members of a group you're in can use the bot within the shared $20. If you add someone in that group to your allowed users list then that person will use their personal budget instead.

as an aside, a limit per group chat would also be nice

fair enough, ugh, for another time 😅

[n3d1117]

Closing this, you can now set BUDGET_PERIOD=daily for daily budgets