Closed krschacht closed 6 days ago
@olimart I mentioned this in a PR, but I’m pulling it out into a real issue. FYI in case you want to tackle it.
Food for thoughts:
ApiKey
with type/provider Claude, ChatGPT,...
& shared
attribute.user's key || shared ApiKey
This implies:
admin/api_keys
controller & views to manage keys
admin
attribute to the very first user created to restrict keys managementApiKey
modelI like how you’re thinking. What you laid out makes sense as the place to end up. My inclination would be to get there in a least a couple PRs. One PR could lay down the core logic and just use an ENV as the fallback “shared key” and then a second PR could implement the new admin views & routes.
Want to take this one?
Has this got anywhere? More api-key configurations coming with my next changes, for https://github.com/AllYourBot/hostedgpt/issues/341
Nope, no progress on this one so you're safe to changes in this arena
This is interesting for our use case where we'll have a handful of students who are learning about AI chat interfaces. We'll be providing the API keys and yet we want to be sure we can keep usage within reasonable limits on a per-user basis.
As for cost tracking, is that something like keeping track of token usage per conversation? It seems like it'd be hard to do per message, especially if there's a moment where HostedGPT is unable to send all of the preceding messages, but instead sends summarized messages or perhaps only the last N messages of a conversation to keep the token count underneath the max_tokens
limit. Or were you imagining tracking in terms of USD based on a pricing table?
Also, fwiw, we have a very small change that uses two env variables: DEFAULT_OPENAI_KEY
and DEFAULT_ANTHROPIC_KEY
. I suppose those names could be changed. We've also got REGISTRATION_FEATURE
set to false
to ensure we're in control of user account creation. I'd be happy to open a PR that adds those couple of ENV variables.
@matthewbennink My initial thought on cost tracking was:
Message.created_after(Date.beginning_of_month).sum(estimated_cost)
message.save
we count the total number of tokens received back from the API (I think it's just Tiktoken.count(message.content_text)
and += to add that onto the preliminary estimated_costClosing this and opening #402 as a placeholder for token cost tracking
Some people want to setup an instance of HostedGPT and let a couple special people use it. In this scenario, they don’t want everyone to have to enter their own API keys they want to let them just piggy back on some master API keys and foot the bill. In this scenario, my assumption was: we could read the API keys out of some central config.
If we implemented it this way then we would want to allow trusted people to register but prevent random people from registering. Maybe there needs to be an option to set a registration code? And, when I tell a trusted friend my registration code and they sign up, if that friend goes to their account settings how should the API key fields behave? Maybe the fields should still be there so they can input their own keys instead of using the master ones. Probably that’s the case.
Note: We’ll need some way to lock down user registration once this is implemented.