Improve performance by adding a 2 layered cache

[ddragosd]

Use this cachemanager module and configure a 2-layered cache in the Gateway:

1. layer 1 : local cache in lua VM using lua_shared_dict
2. layer 2: Redis cache

The README of the repo has this documented but I'm sharing a simplified version here: For setting up we can add the code bellow to routing.lua and other places that use Redis.

            local cache_cls = require "api-gateway.cache.cache"
            ngx.apiGateway.request_cache = cache_cls:new()

            -- define a local cache with a max TTL of 10s
            local local_cache_max_ttl = 10
            local local_cache = require "api-gateway.cache.store.localCache":new({
                dict = "cachedrequests", 
                ttl = local_cache_max_ttl
            })

            -- define a remote Redis cache with  max TTL of 5 minutes
            local redis_cache_max_ttl = 300
            local redis_cache = require "api-gateway.cache.store.redisSetCache":new({
                ttl = redis_cache_max_ttl
            })

            -- NOTE: order is important as cache stores are checked in the same order
            ngx.apiGateway.request_cache:addStore(local_cache)
            ngx.apiGateway.request_cache:addStore(redis_cache)

Then when storing something in cache use:

ngx.apiGateway.request_cache:put(key, value)

And when reading something from the cache use:

ngx.apiGateway.request_cache:get(key)

The lib will take care of using both cache stores appropriately.

[taylorking]

Awesome thank you

[seriousme]

If you use the local cache, do you still need the Redis cache ? Or can we (optionally) skip that ? (and maybe add a few couchbase slaves if needed)

For large installations Redis might still be a requirement, however for smaller installations the smaller the number of technologies to be managed the better.

[mhamann]

The ultimate goal is to make the storage layer pluggable so that you can plug in any backend you like as long as you code to the appropriate interface spec.

[mhamann]

@seriousme checkout v0.9.0, which includes the pluggable storage layer.

We have plans for a few additional drivers that will be officially supported (Cassandra, local filesystem, and in-memory for now). If there are others you think may be useful, feel free to open an issue. We'd also welcome some PRs in that area ;-)

[seriousme]

I have to admit that I'm getting a bit lost here after reading: https://medium.com/openwhisk/uncovering-the-magic-how-serverless-platforms-really-work-3cb127b05f71

Where does the APIgateway fit in the picture depicted in this article ? I thought it was providing the Nginx service at the top, but he picture shows no Redis ?!?

[rabbah]

An API gateway is a general framework for managing APIs. It is not "serverless" specific. Relative to the diagram in the article you linked, it's a layer on top of the nginx router that's part of openwhisk. They serve different purposes. Yes, you can conceive of collapsing the routers. But if these are separate services, the separation is usually necessary.

[seriousme]

Thanks for the explanation ! I know what an API gateway is supposed to do, just couldn't fit how it was specific to openwhisk. Stupid question maybe, but if the openwhisk API gateway is not specific for openwhisk, why is it then labeled as openwhisk-apigateway ? If it is supposed to be part of the openwhisk ecosysteem it would seem more natural to me to use couchdb as (caching) datastore instead of redis.

Just being curious.

[rabbah]

Thanks for clarifying the question - I don't see it as openwhisk specific but others may disagree. Here's an example plugin for Kong: https://getkong.org/plugins/openwhisk/.

[mhamann]

I agree--designed to be a generic gateway--somewhat of a "competitor" to Kong, I suppose. However, it was designed to fit nicely with Whisk, with built-in policies that work well with Whisk actions whether they be "web" actions or "standard" ones (see request mapping).

We probably should build a couchdb connector though, as that would tie nicely into the default OpenWhisk build.

[ddragosd]

Actually, currently, it seems to be a tight coupling between Openwhisk and this API GW in the sense that the GW is not readily isolated in order to make other implementations pluggable. See core package which is coupled with GW's Management API, and the cli. So whether the GW is OW specific or not is one aspect, and whether OW is currently specific about which GW to use, in this sense, is another aspect.