Add test to BasicActionRunnerTests to confirm runtime supports global state

upgle commented 5 years ago

Environment details:

native ubuntu

Steps to reproduce the issue:

I've tested whether global variables could be reused when container was reused. The test results show that only nodejs runtime can reuse global scope variable.

The AWS Lambda service is not stateless for all runtimes. (it means an action can reuse global variable if container is reused) Is there any OW spec for a global variable?

1. Javascript

let globalCache = {};
function main(params) {
    globalCache[getRandom()] = getRandom();
    return globalCache;
}
function getRandom() {
  return Math.random();
}

If the container is reused, the global variable is reused.

{
  "0.25746017280185063": 0.7853652208888477
}

{
  "0.25746017280185063": 0.7853652208888477,
  "0.4376083800717476": 0.6122601078429637
}

2. PHP

<?php
print_r($GLOBALS);
if (!isset($GLOBALS['count'])) $GLOBALS['count'] = 0;
function main(array $args) : array
{
    $GLOBALS['count'] = $GLOBALS['count'] + 1;
    return ["count" => $GLOBALS['count']];
}

The global variable is not retained even if the container is reused. (Only returns 1)

{
  "count": 1
}

3. Python

g = 0
def main(args):
    try:
        global g
        g = g + 1
        return {"count": g}
    except Exception as e:
        # please handle exception here
        # you can return or hide an error message
        return {"error": str(e)}

The global variable is not retained even if the container is reused. (Only returns 1)

If you run the same code in AWS Lambda, the count will increase.

{
  "count": 1
}

4. Java

The static variable is not retained even if the container is reused. (counter variable is always 1)

class Global {
    public static int counter = 0;
}

public class Application {
    public static JsonObject main(JsonObject args) {
        Global.counter++;
        System.out.println(String.valueOf(Global.counter));
        return null;
    }
}

log

rabbah commented 5 years ago

There isn't a uniform tests for all runtimes - or a requirement - that this should be supported since actions should be stateless. That said, we did deliberately permit global state carryover in node and I thought also in python.

https://github.com/apache/incubator-openwhisk/pull/2522#pullrequestreview-52263351

rabbah commented 5 years ago

Here's how to do it with python:

> cat t.py
def main(args):
    try:
       if globals().get("global_context") is None:
         globals()["global_context"] = dict()
         globals()["global_context"]["g"] = 0

       g = globals()["global_context"]["g"]
       g = g + 1
       globals()["global_context"]["g"] = g
       return {"count": g}
    except Exception as e:
        # please handle exception here
        # you can return or hide an error message
        return {"error": str(e)}

> wsk action update t t.py
> wsk action invoke t -r
{
    "count": 1
}
> wsk action invoke t -r
{
    "count": 2
}

upgle commented 5 years ago

oh, I forgot checking that global variable is set. python runtime supports global variable as you said.

if 'g' not in globals(): # should be checked that variable is set in globals
    g = 0
def main(args):
    try:
        global g
        g = g + 1
        return {"count": g}
    except Exception as e:
        # please handle exception here
        # you can return or hide an error message
        return {"error": str(e)}

rabbah commented 5 years ago

We should consider adding this py example to the docs although it encourages stateful invocations, it is also pragmatic.

upgle commented 5 years ago

yes we can add this stateful example. (user should know invocation can be stateful to prevent unexpected behavior of aciton code.)

5. Ruby

I'm not sure if the code below is correct because i'm not a ruby expert. ruby also does not seem to support global state.

global_variables.sort.each do |name|
  puts "#{name}: #{eval "#{name}.inspect"}"
end

if !$counter
  $counter = 0
end

def main(args)
  $counter = $counter + 1
  { "counter" => $counter }
end

upgle commented 5 years ago

java runtime also supports global state (using static keyword) per container.

If an error occurs in the action code, global variables are not stored.

Sample code


class Global {}

public class Application {
    private static Global global;
    public static JsonObject main(JsonObject args) {
        if (global == null) {
            System.out.println("global is null");
            global = new Global();
        } else {
            System.out.println("global is not null");
        }
        return new JsonObject();
    }
}

JiniousChoi commented 4 years ago

@rabbah I am considering utilizing a global variable in a python action for a DB connection pool so as to avoid unnecessary overload on my DB system.

My question is: What do you think about this pattern? Is it encouraged?

upgle commented 4 years ago

@JiniousChoi

If you use a global variable, as you said, you can reuse the DB connection pool or use the cache layer.

I think it is more useful to use it if the action code developer understands the global object well. (you should handle an exception, and consider the memory limit of the container. )

There are also some examples of actually using the global object to retain state.

rabbah commented 4 years ago

With the introduction of action loop as the common proxy, we can also close this issue. Other runtimes using the new proxy support globals.

Can try them here https://apigcp.nimbella.io/wb.

I didn't try Ruby but confident it also works because it also uses the new proxy. It's conceivable we can add a new test to the basic runtime test suite to cover this use case now.

apache / openwhisk