Create AWS Billing Alarm By Default

[Miserlou]

This is just a ticket for discussion, it might be beyond the scope of Zappa, but during @daheats's DjangoCon talk on Alexa skills, she mentioned creating Billing Alarms as a fail-safe against massive financial fail - as I've seen massive $$$ fail happen a few times in the wild, it might be useful to set a sane default for Zappa to create during deploy. (Especially with applications that have S3 event triggers that re-upload, as you could easily create an infinite loop.)

[mjschultz]

I'm not sure a general billing alarm makes sense for zappa to create (though I agree that you should set billing alarms on your AWS account!). For example, we deploy 3 zappa lambdas and have a host of other AWS services, no way could zappa figure out a sane value for our billing alarms. (Unless there is a lambda-specific billing metric I don't know about).

I would say a realistic thing would be to create alarms on the invocation count or sum of duration for the zappa lambda function. Since lambdas are charged by duration it seems like a good proxy for a billing alarm? I'm not sure what the config option would be though (enable_alarm, expected_invocations, expected_duration and some math during deploys)?

[Miserlou]

Ah, excellent point, I think that actually makes a good proxy. Yes, I like that more.

[bcongdon]

I'm looking into creating this. I think for now, we can probably limit this to function duration -- instead of creating another alarm for invocations. (Money wise, compute-time overshadows request pricing: https://aws.amazon.com/lambda/pricing/)

As far as creating the alarm, I'm thinking that we should set it at hourly resolution. So, expected_duation would be in milliseconds, and expected_invocations would be reqs/hr. Then, we'd calculate an hourly compute time 'allowance', and set an alarm to trigger if we exceed that.

We can set the defaults to be rather generous, so this doesn't trigger in normal/modest operations. I think the default config should only alarm you when something has obviously gone wrong -- like the s3 infinite loop example.

[kennethreitz]

✨🍰✨

[daheats]

I'm glad that you got billing alarms out of my talk! Hope it helps save some $$!

On Wed, Aug 30, 2017 at 12:12 PM, Kenneth Reitz notifications@github.com wrote:

✨🍰✨

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Miserlou/Zappa/issues/1053#issuecomment-326090321, or mute the thread https://github.com/notifications/unsubscribe-auth/ATdo1b7GK6FXnupcgd7lKST0rDeue8cUks5sdbQbgaJpZM4O2ycb .

-- All my best,

Heather Luna DjangoCon US Sponsorship Chair P. 267-838-7076 @h34th3r329

The DjangoCon conference is organized by DEFNA https://www.defna.org/about/, a non-profit 501(c)(3) organization, so your sponsorship is tax-deductible.

[mdorn]

Any progress on this @bcongdon ? I recently experienced a financial fail (relatively mild, fortunately), and am very interested to see this in Zappa. Also worth considering: a feature to cause the function to self-destruct (i.e. delete itself) if a given invocation threshold is exceeded. For non-mission-critical functions, this could stop the hemorrhaging immediately.