A case for invokers

[scothis]

One of Elafros’ design goals is for the 'same API to handle applications as well as functions.' That API from the platform perspective is the container contract. Elafros does not provide a significant runtime distinction between containers, applications, and functions.

This issue is attempting to define the need, scope and role of an Invoker. It does not define the Invoker contract.

Elafros itself need not provide Invokers, but the platform does need to define a contract for developers and ISVs to create consistent invokers that function authors use to deploy functions. The invoker contract defines how a function interfaces with the rest of Elafros.

Function → Application → Container

There is an inverse correlation between simplicity and responsibility as the programming model shifts from Functions to Applications to Containers. A Container is the universal runtime within Elafros, any workload that Elafros can manage ultimately runs as a container. While containers provide the most flexibility, they also impose the greatest responsibility on their authors.

Functions are simpler to implement, while also being the most constrained runtime. A Function practices Inversion of Control as its lifecycle is managed externally. How the function is exposed as a Container is the role of the Invoker.

Application = Function + Invoker
Container = Application + Dependencies

Elafros has good support for Containers and Applications, however, Functions are under defined. A Function cannot run on its own, but needs an Invoker to handle application concerns like listening on a port for requests and mediating network requests into meaningful arguments for the function.

Invoker role

For a simple JavaScript helloworld function:

module.exports = name => `Hello ${name}!`

The invoker assumes many responsibilities on behalf of the function. For example, a node invoker would:

• provide a runtime (Node)
• pick a version of that runtime (lts)
• define an entry command (node ./invoker/server.js)
• require the function (const fn = require('../function/'))
• start a server to receive platform requests (binds to port 8080)
• for each incoming http request:
  • convert the request entity into a type appropriate for the function input (Content-Type: text/plain)
  • invoke the function with the input (fn(input))
  • capture the function output (**const output =** fn(input))
  • convert the output into a response entity (Accept: text/plain;q=0.8, application/json;q=0.5, */*;q=0.1)
  • capture errors (try { … } catch (e) { … })
  • send the http response (200)

Each of these steps is an opportunity for an Invoker to provide custom behavior that grants a new capability to the function.

• a different runtime could be used (chakra vs v8)
• a newer (or older) version of node could be used
• lifecycle hooks
• custom mime marshalers
• async functions
• ...

Function ambiguities

The Invoker contract’s role is to define how the invoker fits into the broader Elafros ecosystem. Without an Invoker contract, we have already seen multiple mechanisms for invoking a function over HTTP. Consider these two models, both of which are used by current samples:

GET /execute?param=value

POST /
Content-Type: application/x-www-form-urlencoded

param=value

Which model is correct? How does a caller know which model is used? It’s critical for external callers of a function to be able to invoke Elafros functions in a standard way.

Future proofing

For request-reply functions over HTTP the contract may be nominal as the HTTP spec defines most of the norms, however, as the platform gains additional capabilities (such as event streams or protocols beyond HTTP, such as gRPC https://github.com/elafros/elafros/issues/707 https://github.com/elafros/elafros/issues/813) invokers will need additional guidance to insure the platform can interact with a function in a standard way.

Developer experience

In attempting to think through how functions could work in Elafros. I took a sample function and deployed it via an Elafros Configuration which uses the buildpack BuildTemplate configured with a custom buildpack that implements a Node.js Invoker. The buildpack takes the function source and produces a container that conforms to the container contract. The developer is able to push a function to GitHub and Elafros is able to build and run the function without further intervention.

While the function is callable, it suffers from the same ambiguities described above.

Prior art

• AWS Lambda Execution Environments
• Fission environments
• Google Cloud Function variants (HTTP vs Background)
• OpenWhisk development environments
• riff invokers

/area API /kind doc

[markfisher]

/area build

[evankanderson]

Thanks for the clear writeup. Internally, we've been calling this a "function framework", which adapts the external container and eventing contracts to a language-specific function API:

As I mentioned on slack, I think there are 3 pieces:

1. Adapt user code/function to be able to run as a container This is setting up a main(), starting express, possibly handling health checks.
2. Adapt event delivery to be delivered to user code This is handling inbound events, unmarshalling them into native objects, and invoking the user's function.
3. Provide build & runtime tooling for the above (which moves things past "just a library") This is the build template, any sort of auto-detection and library loading, and performing the inversion of control.

I think the contract you're asking about in "Function ambiguities" should be the same as the output of the event API (elafros/eventing), but I don't know that the event delivery mechanism(s) are standardized yet. I may also be misunderstanding either the problem, or the desired solutions.

@ultrasaurus @markfisher @inlined @vaikas-google have been working on the eventing machinery, so they may have more to say, including whether the eventing delivery mechanism is the right tool to apply to the invoker role as described here.

[inlined]

I've been trying to get some initial steps going for this. We have a PR out for (un)marshalling CloudEvents and hosting webhooks that receive them. I think this is necessary but not sufficient for the CNI of a Function. I hope to get a rough foundation in elafros/eventing that acts as a brain dump of sorts from the Google Events team and then we can start talking about possible features and prioritization. e.g. I definitely want payload (e.g. API version or verbosity) and transport (HTTP, gRPC, etc) negotiation in the long-run, but I'm not sure where that would fall in our set of priorities.

[markusthoemmes]

Great writeup @scothis.

Over at OpenWhisk, there is a contract in place that strictly defines the function's input/outputs as JSON. A function must take JSON and must return JSON (most apparent in typed languages like Java).

The platform mostly resembles @evankanderson's picture, where the user can provide their own container (called "blackbox" at OpenWhisk) and thus is responsible to have the whole HTTP communication in place etc. etc. She also can "just" provide the function, where the function-framework ("runtime" in OpenWhisk) adapts everything as necessary.

However, there might be some kind of middleground. In OpenWhisk exist "web-actions". Those take a raw HTTP request and marshall that into a JSON object. The container can then respond with JSON object of a pre-defined structure to control how the HTTP response to the user should look like. We've certainly seen users wanting to have full control over the incoming request (this might be raw HTTP, but could also be something different like gRPC), while still having the niceness of not having to build a whole container but being able to write a function as they would usually.

[markfisher]

@markusthoemmes I'm glad to see that this issue is initiating comparisons of the contracts for existing FaaS implementations! I believe Elafros provides a great opportunity to create a common model - at least at the K8s resource level, if not more.

In Elafros, it would seem that the cases where a developer wants full control of the HTTP request or the container itself ("blackbox") would be handled as application or container workloads rather than functions. One of the nice things about Elafros is that it provides consistency across the spectrum such that desirable characteristics normally associated with FaaS (such as the ability to scale 0-N-0) are available regardless of workload type.

As for the JSON-only case, it might make sense to have a more general contract but to allow certain invokers to indicate the MIME-types they are able to handle. That way, a particular invoker implementation could be opinionated about JSON, but others might accept a broader range of types.

The gRPC option is something we on the riff team definitely want to see in Elafros. We use that in our current invoker contract, and it allows us to support streaming use-cases as well. That's why we've also created #813 (there again, one the nice thing about Elafros is that the gRPC stream support need not be limited to the realm of functions).

[mattmoor]

cc @swalkowski

[mattmoor]

cc @steren

[inlined]

The CloudEvents spec requires that all implementations handle JSON as the Event envelope, though it supports double-encoding for mixed content types (e.g. the JSON-encoded envelope has a contentType: "application/xml" property and a JSON-encoded XML string for the data property).

There's some early (but rapidly solidifying) thoughts on a webhook standard that is more rigorous than the mere HTTP transport of CloudEvents. The spec calls for negotiation over the OPTIONS method; we could push that the actual event data be JSON with something like acceptDataType: "applicaton/json" (presumedly acceptType: "application/json" would apply to the whole envelope)

[evankanderson]

Ref for the OPTIONS method: https://tools.ietf.org/html/rfc7231#section-4.3.7

I think there's some interesting possibilities about enabling an upgrade from HTTP/1.1 (a lingua franca for containers) to something richer, such as gRPC. To me, this feel like an optional contract between the event broker and the function runtime -- Elafros routing should get out of the way as much as possible[1] to allow the broker to deliver requests to the container with appropriate framing.

[1] With the caveat that we'd like the routing to be able to provide visibility into offered and consumed work to inform the autoscaler.

[mbehrendt]

in this issue the terms function, application and containers are used a lot, in various ways.

I think it would be helpful to me have them more crisply defined. E.g. @markfisher mentions above to use applications or containers rather than functions for certain purposes. In that context it's not clear to me what the actual difference between those concepts is. Eg are we having a mental model where you can't distinguish an app, container or function anymore once they're deployed, i.e. depending on which model you as a developer chose, you only have to provide thee top part of the stack (->function) or you can also provide lower parts (->container), but the end product looks the same. Or is it that the northbound interface of a function, app and container are actually different?

From my perspective, a function is a snippet of code following conventions like what @markusthoemmes described above. A container could follow the same northbound conventions, but you as a dev don't provide just a snippet of code, but a full container image. An app feels like some middle ground, but it's not 100% clear to me what the exact delta would be.

This is probably/hopefully just a newbie question and it's already covered and I just haven't been able to find the doc, so please bear with me :-) . Any pointers/perspective are very much appreciated.

[deissnerk]

@inlined Similar to the CloudEvents spec for http and web hook we could propose CloudEvents via protobuf and gRPC.

[scothis]

in this issue the terms function, application and containers are used a lot, in various ways. I think it would be helpful to me have them more crisply defined.

I'll start with my definitions for containers, applications and functions. Others should refine them or propose alternatives as needed.

Container. A fully packaged OCI (Docker) image that conforms to the Elafros Runtime Contract. Containers are the output of the Elafros Build system, or may be provided full formed.

Application. Source code that is transformed into a Container by the Elafros Build system. An application runs on Elafros via a Container. Applications host an HTTP service by starting a process that listens on 8080 and have full control over the HTTP request/response.

Function. A single purpose unit of logic. Like an Application, a Function is transformed into a Container by the Elafros Build system. A function invoker handles binding the function to HTTP.

By these definitions, a function and application are similar. However, because a function has a single responsibility, it needs something to expose it to the external world. Unspecified by this definition is the form of the HTTP traffic the function can handle (since we haven't reached consensus). Also specifying HTTP since that's what the runtime contract specifies, not to exclude other protocols down the road.

[jchesterpivotal]

I started trying to define contract layers last week. One thing that seems to show up is that different contract layers apply at different points in the lifecycle of functions or apps.

Contract Layer	Build Time	Register Time	Launch Time	Invocation Time
Source code & language runtime	Buildpacks detect intent and provide runtimes			Published API for functions (earlier recognised by Buildpacks)
Envelope				As received by sidecars and gateways, then passed in via stream or attached volume
Transport				Streams, HTTP via sidecars and gateways
Function / Task / Resource / App / Universal Turing Machines		Information recorded by platform	Configuration passed in	Inputs passed in, outputs collected
Elafros Runtime	FS layout		Entrypoints, permissions etc
OCI	FS layout	Image metadata	OS container config

Building from @scothis 's definitions above, I'd say that the distinction between apps and functions shows up mostly in the number of contracts which are defined. For functions meant for a FaaS, all contracts are fully defined -- convention over configuration taken to its logical conclusion. For apps, there is some flexibility until you hit the Elafros Runtime contract, at which point the platform stops bending.

[jchesterpivotal]

I should add that Build doesn't necessarily have to be Buildpacks here, I just find that easier to think of because of my experience.

[mbehrendt]

thx a lot for sharing your views -- this is very helpful.

Below I tried to capture what I think is the 'traditional' view of these concepts: Function -- code (provided as a set of files or zip) that has a JSON-in/JSON-out interface App -- code (provided as a set of files) that talks HTTP-in/HTTP-out Container -- any kind of code or binary that talks essentially any protocol in and out. I think container is slightly different beast than app and function, since it's a packaging technology, much less of an programming interface contract. In other words -- a container can also be used to ship a function or an app.

A big attractor for serverless/FaaS has been the pay-by-the-request billing model. Initially that was scoped to functions only, but based on what we're seeing in terms of customer demand is that attribute should not only be available for JSON-in/JSON-out code, but also for what we traditionally call apps or just any binary code that could be packaged in a container.

Based on that, I think there is the function notion of how you pay and scale (by-the-request) and there is the function notion of JSON-in/-out as the protocol convention, and often these 2 get conflated. As we loosen the latter one, I can imagine the former one is going to be more present. Based on that, I'm wondering how much sense the distinction between app & function really makes, from a strategic PoV? Isn't an app also a function, just with a less strictly defined I/F? And isn't the content of a container also just a function (if executed in elafros), and the container happens to be the packaging format vs 'just' uploading the code?

Thoughts?

[scothis]

Thanks @mbehrendt

Part of the fun will be deciding what these terms mean within the context of Elafros. Here's where I think we agree so far:

• code is the core deployment unit for functions and apps, containers are pre-packaged deployments
• applications are driven by http (or another protocol)
• functions are transport agnostic, driven by data (events)
• it should be possible to bill by invocation
• a container can contain an application or function
• the distinction between apps and functions are a bit murky in Elafros today (hence this issue)

Areas we don't fully align (yet):

• JSON is one form of data that can drive a function, but not exclusive
• Elafros should not impose a billing model, but should enable ISVs/operators to bill as they desire
• an application is not necessarily a function with a less strictly defined interface, but a function could be deployed within an application (via an invoker).

[mbehrendt]

thank you @scothis -- this is a good list of attributes to work through.

Below I captured some comments:

applications are driven by http (or another protocol) functions are transport agnostic, driven by data (events)

hmm....if apps can support http or another protocol, and functions are transport agnostic , it feels like there is some commonality. We might want to have a brief chat today to distill out the delta you have in mind.

JSON is one form of data that can drive a function, but not exclusive

along the lines of my previous comment, I can see functions also supporting something else than JSON

Elafros should not impose a billing model, but should enable ISVs/operators to bill as they desire

I agree that Elafros should not impose a billing model. The reason why a discussion about billing models is still very important, is because those models are going to drive technical decisions on the next level of detail.

an application is not necessarily a function with a less strictly defined interface, but a function could be deployed within an application (via an invoker).

that is an interesting comment, would like to drill a bit deeper on it. I've been viewing an app as a peer to function so far, but what you're saying above implies that an app can contain a function. I'm not sure I have my head around the semantics of that yet, so would be great to chat about it in more detail later today.

[evankanderson]

To extend @scothis comment:

If the eventing contract specifies e.g. HTTP as a fallback delivery mode with additional upgrades (e.g. to gRPC or something else(), it should be possible to build an application which implements the eventing contract as part of its overall interface.

It should also be possible for an application (or a container) to implement multiple event delivery endpoints in a single container, which would give you a multi-functional container.

One distinction I'd draw from @scothis bullets: in my mind, container images are the core type ingested by the serving stack; the build CRDs provide a consistent and canonical way to convert application or function code to a container. Once the container image is built, the remainder of the system shouldn't need to distinguish between the two.

[scothis]

One distinction I'd draw from @scothis bullets: in my mind, container images are the core type ingested by the serving stack; the build CRDs provide a consistent and canonical way to convert application or function code to a container. Once the container image is built, the remainder of the system shouldn't need to distinguish between the two.

@evankanderson we're in complete agreement.

[takidau]

Overall this makes sense to me, and I like how it clarifies the relationship of functions to containers.

One thing I'd like to confirm for my own understanding, that I think @jchesterpivotal may have answered above, is how this all relates to various types of data sources in an eventing context. Am I right that if you're consuming events from a source like Kafka where there's a non-trivial protocol to follow for consuming the data, the logic implementing that protocol (or using the appropriate client lib) lives in a sidecar that then feeds the Invoker via HTTP or gRPC? And if so, is the intent that such sidecars are pluggable/swappable, presumably via a CRD definition of their own? I feel like that's the spirit of things here, but don't see it detailed out beyond @jchesterpivotal's brief reference in the contract layer table.

[scothis]

@takidau I've intentionally avoided talking in this thread about how riff works today. But since you asked, in riff messages are consumed by a function-sidecar from Kafka and then sent to the invoker over gRPC, which then invokes the function. The sidecar is able to send traffic to the invoker over either HTTP or gRPC, the function specifies which protocol it wants.

We're currently exploring a spike to separate the function sidecar from the function/invoker and instead have a pluggable CRD-defined broker (e.g. Kafka) that sends events to the function via envoy. This would be more in line with how Elafros works today, where the riff Function CRD is replaced by Elafros Services.

[jchesterpivotal]

@takidau - I think @scothis has pretty much hit it on the head for what will need to happen in the nearterm. I also have an optimistic eye swiveled towards a future in which Concourse can use Elafros primitives for its runtime (there's already work towards k8s, so why not Elafros?). For that to happen, being able to define sources/sinks of data independently of apps/tasks/functions is a necessity.

[scothis]

I took a pass at drafting the Function Invoker Contract based off the conversation in this issue. Comments and suggestions are welcome.

[evankanderson]

Awesome, thanks @scothis. I took a look, and this looks like a good start, if I'm interpreting the doc correctly. I assumed that the Invoker was the same role as we've been calling Functions Framework. I'm not attached to Functions Framework, so if Invoker is the same thing, feel free to suggest that as a replacement name in the glossary doc.

[scothis]

I'm not particularly attached to the term 'Invoker'. The 'Function Framework' sounds like a single thing that you use to create functions, versus what I'm imagining is an ecosystem of competing frameworks/invokers for functions. The ecosystem would be similar to how the BuildTemplate CRD is an extension for ways to create builds.

[mattmoor]

I haven't heard much contention in this space in a while, so I think that folks generally agree on the seams between these things. If not, feel free to /reopen.