Make spend limits "budget suggestions" rather than hard limits

[novoselt]

The goal of these limits is to prevent users from spending more than intended accidentally, because that is bad and unpleasant. But it is also unpleasant to accidentally shut down or delete something, while there are money on the account. In fact, it is way worse to loose a disk with data I was creating for several weeks or computations that were supposed to get done over the weekend to be ready for Monday presentation.

There is a hard limit - the amount of money put into CoCalc. Other spending limits should be a suggestion, at least by default. Admins may want hard limits imposed on particular projects/compute servers, which is a separate issue to deal with.

When a spend limit is hit:

• display a noticeable banner with a link to the limits page - the user has to adjust limits to make the banner go away, do NOT add "Dismiss" button
• send an email
• that's it! Do NOT take it into account when approving any actions.

[williamstein]

What are your thoughts about one single global limit, versus a dozen limits for different things?

[novoselt]

I am very much against configuring a separate limit for every single bloody model! Having separate limits for LLMs/network/compute servers in theory makes sense, but I think won't really matter all that much. If somebody does spend $100 on network (which is very unlikely) they will learn their lesson quickly, I suspect. So a single limit should be fine.

What is much harder for me to decide, is the length of the period which is limited. I think there are arguments both for daily and for monthly limits. So perhaps we should support both or at least let the user choose. But I think two limits are easier to understand and use than a single one with the length selector.

[williamstein]

What is much harder for me to decide, is the length of the period which is limited. I think there are arguments both for daily and for monthly limits. So perhaps we should support both or at least let the user choose. But I think two limits are easier to understand and use than a single one with the length selector.

For what it is worth, on Google cloud (and cocalc right now) there is only a monthly limit. This aligns with the fact that billing is done monthly.

[novoselt]

Then let's have a single monthly limit and revisit it if there is any demand from actual users!

[haraldschilly]

My "perfect world" wishlist:

• IMHO a single spend limit is all I would need. Someone who just wants to use LLMs, should set a low limit – others with compute servers and PAYGO set a higher limit.

• If an LLM calls' maximum cost estimation or attempting to run a compute server exceeds the remaining budget, there should be an easy way to bump up the limit. e.g. buttons "+25%, +50%, +100%" or so, i.e. it depends on what I've set as a limit, and it allows me easily increase it by a comfortable margin (or just enter a number, of course).

• Email notifications when it is about to be reached (maybe 75%?) sounds great, and also when it is hit and it needs my intervention to avoid something to be deleted or shut down.

[williamstein]

To get your 'perfect world', we should have a checkbox about whether or not it is a soft or hard limit. soft = just notifications; hard = could stop/delete stuff.

[williamstein]

https://cloud.google.com/billing/docs/how-to/budget-api-overview

[novoselt]

So maybe there can be single easy to understand and configure budget limit for mere mortals and a powerful flexible API for those who really need it? CoCalc and Google users are very different groups.