Add ability to keep functions "warm" for a period of time

[pillowsoft]

Description

Cloud functions are typically not true lambda functions, as the cost of starting them can be high. Most will start and then run for a period of time (say 3 - 10 seconds), accepting multiple REST calls in that time period. In the current emulator, the functions are restarted on every call (at least in REST mode, I have not tried RPC). Is there any flag to tell a function to stay warm for some period of time, to more accurately model cloud hosted functions?

Thanks!

[jmdobry]

Are you utilizing the temporary file system of the cloud hosted functions? Or otherwise relying some state to remain the same between function calls?

[pillowsoft]

I'm just using the memory in the functions process/container. In the cloud, a function can handle multiple requests before it may be evicted. Launching a function, requiring it, and having the javascript parsed, each time is too much overhead, so most functions live long enough (speaking here generically for most of the various cloud function providers out there) that they may receive several calls. This allows one to amortize things like connecting to external databases, initialization startup, caching results from previous lambda calls, etc. over several calls.

One must still ensure that the function does not rely on anything that might not be available to a function started in another container as scaling occurs, but by caching connections and simple data locally, cloud function performance can be improved.

The current cloud function emulator is effectively restarting the "container" on each call, and I believe, incorrectly modeling what happens in true cloud functions, IMHO.

[jmdobry]

Interesting, can you provide some sample code showing how you do some of this caching? It would help me understand how to better emulate the cloud hosted functions.

[pillowsoft]

Sorry for the delayed reply. Here's some psuedo-code:

if (globals.databaseConnection === undefined) {
     globals.databaseConnection = createADatabaseConnection();
}

// now use the connetion
globals.databaseConnection.lookUpSomeId(someID);

In this case, we are just "caching" the database connection, but you can do much more and by doing this, you don't incur the overhead on every single call (just when the function first "cold starts").

[jmdobry]

Will be available in the 1.0.0-alpha.7 release.

[pillowsoft]

Thanks!! :clap:

[jmdobry]

Note, in 1.0.0-alpha.7 you currently have to delete the function and re-deploy it for any changes to take affect, as consequence of the architecture change. In 1.0.0-alpha.8 you'll be able to re-deploy the function without having to delete it first. In 1.0.0-alpha.9 changes should be picked up automatically.

[VikramTiwari]

@jmdobry On the opposite end of this. Is there a way to keep cloud functions (deployed) always warm? I understand that this might defeat the purpose but is there a better strategy/flag to do this rather than pinging the function after 10 minutes?

[jmdobry]

Functions in the Emulator? Or functions deployed to production?

[VikramTiwari]

The ones deployed on production.

[jmdobry]

I don't know of a better strategy :/