search

the-full-stack / website

Source for https://fullstackdeeplearning.com
1.13k stars 203 forks source link

Added cloud GPU options for beam.cloud #75

Closed mernit closed 1 year ago

netlify[bot] commented 1 year ago

Deploy Preview for comfy-licorice-ff5651 ready!

Name Link
Latest commit 30335bf85bafbb164f78286f46f4a83f8235d8fd
Latest deploy log https://app.netlify.com/sites/comfy-licorice-ff5651/deploys/649c8b7b69649200083b0bf8
Deploy Preview https://deploy-preview-75--comfy-licorice-ff5651.netlify.app/blog/posts/running-llm-glm-130b
Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

charlesfrye commented 1 year ago

Thanks for the contribution, @mernit!

Looks like you're affiliated with Beam, so I wanted to confirm one thing about your pricing.

You indicated "No" for the "Idle Time Charged?" column. We include boot time and time between requests as "idle time". Reviewing your pricing docs, it looks like you don't charge for boot time but do charge for warm containers awaiting requests (according to the documentation for keep_warm_seconds).

Compare that with Replicate, which only charges for prediction time, even though they keep containers warm between requests. cf also #76.

mernit commented 1 year ago

Hey @charlesfrye 👋 -- sorry for the late reply.

We actually don't charge for boot time or idle time. The keep_warm_seconds field is an optional parameter to specify how long users want to keep their containers warm, sometimes for an infinite amount of time (essentially, this can be used to disable scale-to-zero).

If a user keeps their container running for 1 hour after their last request, we'll have to charge for that. But if this optional argument isn't used, users won't be billed for time between requests.

charlesfrye commented 1 year ago

Thanks for getting back to me!

As I understand it, Replicate does not charge for time while containers are warm, but it also does not give users the ability to configure that setting.

It seems like this property of pricing is too complex to be squashed down to a Boolean -- at least until terminology is settled -- so I suspect it'd be better put in the notes above, perhaps with some styling to make sure it's noticed.