williamstein
commented
1 year ago are there any estimation on roughly costs coming to us?
First let me note that openai does an excellent job letting you track total usage each day on their website, and also they enumerate each time the API was used. CoCalc itself also has a table that tracks usage, which you can access via postgresql and/or making a file usage.cocalc-crm as an admin.
With the current integration, a good rule of thumb for your monthly bill is to take the number of people who actively use ChatGPT each day via your cocalc and multiple by 25 cents. So if you have 100 users, the bill might be about $25/month total for all of them. ChatGPT 3.5-turbo is a very good value in terms of cost. This isn't counting any bandwidth charges you might incur from your host.
The cost would be massively higher if CoCalc automatically ran ChatGPT in a similar way to GitHub CoPilot or noteable or other integrations. In CoCalc so far we have made the decision that any time somebody uses ChatGPT, they do so very explicitly, and they have a good sense of exactly what context will be sent to ChatGPT. This is arguably better for privacy, and also strikes a balance toward encouraging people to try to solve problems for themselves some before asking for help, which seems reasonable since many people using cocalc are students learning something rather than professionals just trying to get through their jobs.
One side effect though is that the costs are less. For example, for CoCalc it's a few hundred dollars a month, which is why we can make it available to everyone for free.
I do plan GPT-4 integration as an option for users somehow in the near future. That costs about 22x as much per request, and takes ~3x as long (there's a lot of statements online saying "GPT-4 is faster", but they are incorrect). With GPT-4 on cocalc, the model will be that to use it, you have to explicitly enable it and have a credit card on file, and you get charged for your usage at the end of the month. I have it enabled already for admins, and often I just prefer the faster GPT-3.5 results, and for many coding tasks GPT-3.5 is fine; however, for some tasks GPT-4 is better, and then I explicitly use it. I haven't thought at all about how to support GPT-4 for cocalc-docker, but a simple option might be just making it an on/off option in admin settings for all accounts, but still requiring users to explicitly select GPT-4 when they want to use it (i.e., it isn't the default). Of course, this could increase your bill by a lot.
We are planning to enable in out docker instance the Chatgpt, are there any estimation on roughly costs coming to us? mainly our users are using notebooks and latex. Thank you beforehand.