Feature: keep_warm can take integer values

[mcrowson]

The keep_warm feature is a true/false and it can keep your zappa application warm on 1 concurrent lambda. It would be nice to pass the number of concurrent lambdas you would like to keep warm.

Expected Behavior

keep_warm takes a value, such as 100, and keeps 100 lambdas warm for you.

Actual Behavior

keep_warm just takes true/false and either keeps 1 lambda warm or 0.

[Miserlou]

I don't think that it really works that way. If it does, I don't know how to keep that many warm simultaneously, I thought it was sort of an all-or-nothing deal. There is the keep_warm_expression for setting the rate.

[mcrowson]

I haven't looked at the code yet, but my gut is telling me to do it with multi threading lambda requests with a slight time.sleep so that you know each thread is hitting a new lambda.

Sent from my iPhone

On May 14, 2017, at 8:11 AM, Rich Jones notifications@github.com wrote:

I don't think that it really works that way. If it does, I don't know how to keep that many warm simultaneously, I thought it was sort of an all-or-nothing deal. There is the keep_warm_expression for setting the rate.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[mcrowson]

Another approach would be to have the keep_warm handler recursively call keep_warm lambda handlers with the requestResponse event type until we reach the desired number.

[wagmiwiz]

I am doing this at the moment by calling several async @task functions at the same time rather than using threads. They all hit a /pinglike endpoint in the app (via API Gateway). In this endpoint there is a fake wait of a few hundred ms. This ensures that more than one Lambda is instantiated as the requests can't all be served by the same lambda at the same time. I don't know how else to 'force' a cold start of more than one container.

The other reason this goes all the way via an API call rather than just a function is that I also had some non-conclusive evidence that a scheduled keeper warmer keeps a different container warm to the one that an APIG request would hit (i.e. even though keep warm just ran, an APIG request would cause a cold start). Related to: https://github.com/Miserlou/Zappa/issues/418

[matse004]

see also an attempt here: #1790

[tim-schier-frontier]

There is a way to spool up a guaranteed number of Lambda instances, but it doesn't solve for the inherent Lambda cost of n+1 cold starts. To run n Lambdas, you need to have n concurrent Lambda requests executing at once. To guarantee you have n Lambdas executing, you need to kick them off not as the default async execution requests (i.e. kick-off and forget), but synchronous ones (i.e. kick-off and wait for the response).

I wrote a little function that invokes itself up to n times, each time waiting for the subsequent invocation to finish before it finishes itself (it's recursive).

The cost of doing this, of course, is the same cost that exists with Lambda across the board. If all your n warm Lambda instances are busy, and you get another (e.g. a 'live') request for another Lambda at just the right moment, then you're going to see a cold-start. This recursive method guarantees that n Lambda instances will be busy while the recursive warming function is running; same as the existing keep-warm method guarantees that 1 Lambda instance will be busy while the current keep-warm function is running. While the recursive warming function is not running however, you now have n warm Lambda instances ready to go.

I don't believe there's a way to solve the n+1 cold start; seems to me it's a fundamental limitation of Lambda today.

The problem with the approaches earlier in this thread (using sleeps) is that as n increases your likelihood of re-using an existing warm Lambda goes to 1. If you increase your sleep, you occupy your Lambdas for longer (increasing the likelihood of n+1 cold start); if you decrease your sleep, you increase the likelihood of re-using a warm instance and you won't have n warm instances at the end. The recursive approach means the function warms up n Lambda instances, but only occupies them for the minimal period required.

I've implemented the recursive idea and it seems to do the trick, but keen for feedback on the premise.