search

redhat-developer-demos / faas-tutorial

Java FaaS demos with OpenWhisk and OpenShift
Apache License 2.0
43 stars 29 forks source link
demos faas java openshift openwhisk serverless

readme

= Function as a Service(FaaS) Tutorial // Settings: :idprefix: :idseparator: - ifndef::env-github[] :icons: font endif::[] ifdef::env-github,env-browser[] :toc: preamble :toclevels: 5 endif::[] ifdef::env-github[] :status: :outfilesuffix: .adoc :!toc-title: :caution-caption: :fire: :important-caption: :exclamation: :note-caption: :paperclip: :tip-caption: :bulb: :warning-caption: :warning: endif::[]

(C) 2018 https://developers.redhat.com[Red Hat Developer Experience Team]

//Aliases :conum-guard-sh: # ifndef::icons[:conum-guard-sh: # #]

:conum-guard-java: // ifndef::icons[:conum-guard-java: // //]

// URIs: :uri-minishift: https://docs.openshift.org/latest/minishift/getting-started/installing.html :uri-openwhisk-cli: https://github.com/apache/incubator-openwhisk-cli/releases/ :uri-openwhisk-openshift: https://github.com/projectodd/openwhisk-openshift :uri-openwhisk-repo: https://github.com/apache/incubator-openwhisk :uri-repo: https://github.com/redhat-developer-demos/faas-java-tutorial :uri-repo-file-prefix: {uri-repo}/blob/master/ :uri-repo-tree-prefix: {uri-repo}/tree/master/ :uri-openwhisk-docs-prefix: {uri-openwhisk-repo}/blob/master/docs ifdef::env-github[] :uri-repo-file-prefix: link: :uri-repo-tree-prefix: link: endif::[] == Overview

This tutorial walks you through on how to build a Java functions on a Function as a Service(FaaS) platform https://openwhisk.apache.org/[Apache OpenWhisk].

== Prerequisites

You will need in this tutorial

=== Tools

=== Setup minishift Local development and testing can be done using https://github.com/minishift/minishift[minishift]. Minishift is a tool that helps you run OpenShift locally by running a single-node OpenShift cluster inside a VM. Details on minishift and installation procedures can be found https://docs.openshift.org/latest/minishift/getting-started/index.html[here].

==== Minishift Profile Setup

[source,sh,subs=attributes+]

!/bin/bash

Don't foget to add the location of minishift executable to PATH

minishift profile set faas-tutorial minishift config set memory 8GB minishift config set cpus 3 minishift config set image-caching true minishift addon enable admin-user minishift addon enable anyuid {conum-guard-sh} # <1>

minishift start

minishift ssh -- sudo ip link set docker0 promisc on {conum-guard-sh} # <2>

<1> Some images that are in Apache OpenWhisk Docker hub requires __anyuid__ SCC in OpenShift <2> This is needed for pods to communicate with each other within the cluster (TODO: need to add more clear details here) [IMPORTANT] ==== `minishift ssh -- sudo ip link set docker0 promisc on` command needs to be execute each and every time minishift restarted ==== If your minishift is running OpenShift 3.9.0, you'll need to fix another bug: ---- minishift openshift config set --patch \ '{"admissionConfig": {"pluginConfig": {"openshift.io/ImagePolicy": {"configuration": {"apiVersion": "v1", "kind": "ImagePolicyConfig", "resolveImages": "AttemptRewrite"}}}}}' ---- === Setup environment [source,sh,subs=attributes+] ---- #!/bin/bash eval $(minishift oc-env) && eval $(minishift docker-env) oc login $(minishift ip):8443 -u admin -p admin ---- === Setup OpenWhisk The project {uri-openwhisk-openshift}[OpenWhisk on OpenShift] provides the OpenShift templates required to deploy Apache OpenWhisk. [source,sh,subs=attributes+] ---- oc new-project faas {conum-guard-sh} # <1> oc project -q {conum-guard-sh} # <2> oc process -f https://git.io/openwhisk-template | oc create -f - {conum-guard-sh} # <3> oc adm policy add-role-to-user admin developer -n faas {conum-guard-sh} # <4> ---- <1> Its always better to group certain class of applications, create a new OpenShift project called `faas` to deploy all OpenWhisk applications <2> Make sure we are in right project <3> Deploy OpenWhisk applications to `openwhisk` project <4> (**Optional**) Add `developer` user as admin to `faas` project so as to allow you to login with developer user and access `faas` project [NOTE] ==== You need to wait for sometime to have all the required OpenWhisk pods come up and the FaaS is ready for some load. You can watch the status using watch -n 5 'oc logs -f controller-0 -n faas | grep "invoker status changed"' ==== ==== Verify Deployment Launch OpenShift console via `minishift console`. Navigate to the `faas` project by clicking the name in the upper right corner. A successful deployment will look like: image::readme_images/ow_deployed_success_1.png[OpenWhisk Pods] image::readme_images/ow_deployed_success_2.png[OpenWhisk Pods] [[configure-wsk]] ==== Configure WSK CLI Download {uri-openwhisk-cli}[OpenWhisk CLI] and add it your PATH. Verify your path using the command `wsk --help` The {uri-openwhisk-cli}[OpenWhisk CLI] needs to be configured to know where the OpenWhisk is located and the authorization that could be used to invoke `wsk` commands. Run the following command to have that setup: [source,sh,subs=attributes+] ---- #!/bin/bash AUTH_SECRET=$(oc get secret whisk.auth -o yaml | grep "system:" | awk '{print $2}' | base64 --decode) wsk property set --auth $AUTH_SECRET --apihost $(oc get route/openwhisk --template="{{.spec.host}}") ---- Successful setup of WSK CLI will show output like: image::readme_images/ow_wsk_cli_setup.png[WSK CLI] In this case the OpenWhisk API Host is pointing to the local minishift nip.io address To verify if wsk CLI is configured properly run `wsk -i action list`. This will list some actions which are installed as part of the OpenWhisk setup. If you see empty result, please see <> [TIP] ==== The `nginx` in OpenWhisk deployment uses a self-signed certificate. To avoid certificate errors when using `wsk`, you need to add `wsk -i` to each of your `wsk` commands. For convenience, you can add an alias to your profile with `alias wsk='wsk -i $@'`. ==== === Setup your Development environment Clone the complete project from `git clone {uri-repo}`, we will refer to this location as $PROJECT_HOME through out the document for convenience. == What is an Action ? **Actions** are stateless code snippets that run on the OpenWhisk platform. They are analogous to methods in Java idioms. OpenWhisk **Actions** are thread-safe meaning at a given point of time only one invocation happens. Fore more details refer the official documentation {uri-openwhisk-docs-prefix}/actions.md[here]. === Your first Action Let's quickly create a simple function in JavaScript to see it all working: [source,sh,subs=attributes+] ---- mkdir -p getstarted cd $PROJECT_HOME/getstarted ---- Create a file called `$PROJECT_HOME/getstarted/greeter.js` and add the following content to it: [source,js,subs=attributes+] ---- function main() { return {payload: 'Welcome to OpenWhisk on OpenShift'}; } ---- Create an action called **greeter**: [source,sh,subs=attributes+] ---- wsk -i action update greeter greeter.js ---- Lets invoke the action using command: [source,sh,subs=attributes+] ---- wsk -i action invoke greeter --result ---- The action invoke should respond with the following JSON: [source,json,subs=attributes+] ---- { "payload": "Welcome to OpenWhisk on OpenShift" } ---- == Java Actions === Install Maven Archetype Maven Archetype can be used to generate the template Java Action project, as of writing this tutorial the archetype is not maven central hence it need to install it locally, [source,sh,subs=attributes+] ---- git clone https://github.com/apache/incubator-openwhisk-devtools cd incubator-openwhisk-devtools/java-action-archetype mvn -DskipTests clean install cd $PROJECT_HOME ---- === Your first Java Action Let's now create the first Java Action a simple "hello world" kind of function, have it deployed to OpenWhisk and finally invoke to see the result. This section will also details the complete Create-Update-Delete cycle of Java **Actions** on OpenWhisk. [NOTE] ==== For easier jar names all the examples will be using maven ``. If you generating new project following the instructions just be sure to update the default `` in `pom.xml` to `${artifactId}` to make the command instructions in subsequent section work without any changes. ==== ==== Create Java Action [source,sh,subs=attributes+] ---- cd $PROJECT_HOME mvn archetype:generate \ -DarchetypeGroupId=org.apache.openwhisk.java \ -DarchetypeArtifactId=java-action-archetype \ -DarchetypeVersion=1.0-SNAPSHOT \ -DgroupId=com.example \ -DartifactId=hello-openwhisk \ -Dversion=1.0-SNAPSHOT \ -DinteractiveMode=false ---- ==== Build [source,sh,subs=attributes+] ---- cd hello-openwhisk mvn clean package ---- ==== Deploy to OpenWhisk ===== Create [source,sh,subs=attributes+] ---- wsk -i action create hello-openwhisk target/hello-openwhisk.jar --main com.example.FunctionApp ---- [[action-invocation]] ==== Invoke and Verify the result [[sync-invocation]] ===== Synchronously [source,sh,subs=attributes+] ---- wsk -i action invoke hello-openwhisk --result ---- As all the OpenWhisk actions are asynchronous, we need to add `--result` to get the result shown on the console. Successful execution of the command will show the following output: [[action-response]] [source,json,subs=attributes+] ---- {"greetings": "Hello! Welcome to OpenWhisk" } ---- [[async-invocation]] ===== Asynchronously [source,sh,subs=attributes+] ---- wsk -i action invoke hello-openwhisk ---- A successful action invoke will return an **activation id** : image::readme_images/ow_action_with_activation_id.png[Action with Activation ID] We can then use the to **activation id** check the response using `wsk` CLI: [source,sh,subs=attributes+] ---- wsk -i activation result ---- e.g. [source,sh,subs=attributes+] ---- wsk -i activation result ffb2966350904356b29663509043566e ---- Successful execution of the command will show the same output like <>. ===== Update Update the FunctionApp class response with the String: [source,java,subs=attributes+] ---- response.addProperty("greetings", "Hello! Welcome to OpenWhisk on OpenShift"); ---- Update the FunctionAppTest Test class to match the same String: [source,java,subs=attributes+] ---- assertEquals("Hello! Welcome to OpenWhisk on OpenShift", greetings); ---- [source,java,subs=attributes+] ---- cd $PROJECT_HOME/hello-openwhisk mvn clean package wsk -i action update hello-openwhisk target/hello-openwhisk.jar --main com.example.FunctionApp ---- Successful update should show a output like: image::readme_images/ow_action_update_result.png[] Repeating the <> steps should result in the updated response ("... on OpenShift") like: [source,json,subs=attributes+] ---- { "greetings": "Hello! Welcome to OpenWhisk on OpenShift" } ---- ===== Delete [source,sh,subs=attributes+] ---- wsk -i action delete hello-openwhisk ---- A successful delete should show output like: image::readme_images/ow_action_delete_result.png[] === Web Action **WebActions** allow the OpenWhisk action to be invoked via HTTP verbs like GET, POST, PUT etc. The **WebActions** can be enabled for any **Action** using the parameter `--web=true` during the creation of the action using {uri-openwhisk-cli}[WSK CLI]. [source,sh,subs=attributes+] ---- cd $PROJECT_HOME mvn archetype:generate \ -DarchetypeGroupId=org.apache.openwhisk.java \ -DarchetypeArtifactId=java-action-archetype \ -DarchetypeVersion=1.0-SNAPSHOT \ -DgroupId=com.example \ -DartifactId=hello-web \ -Dversion=1.0-SNAPSHOT \ -DinteractiveMode=false ---- Update the FunctionApp class response to show application's arguments: [source,java,subs=attributes+] ---- response.add("response", args); ---- Update the FunctionAppTest testFunction method with code: [source,java,subs=attributes+] ---- @Test public void testFunction() { JsonObject args = new JsonObject(); args.addProperty("name", "test"); JsonObject response = FunctionApp.main(args); assertNotNull(response); String actual = response.get("response").getAsJsonObject().get("name").getAsString(); assertEquals("test", actual); } ---- ==== Build [source,sh,subs=attributes+] ---- cd hello-web mvn clean package ---- ==== Deploy to OpenWhisk [source,sh,subs=attributes+] ---- wsk -i action update --web=true hello-web target/hello-web.jar --main com.example.FunctionApp ---- ==== Invoke and Verify the result [source,sh,subs=attributes+] ---- WEB_URL=`wsk -i action get hello-web --url | awk 'FNR==2{print $1".json"}'` {conum-guard-sh} # <1> AUTH=`oc get secret whisk.auth -n faas -o yaml | grep "system:" | awk '{print $2}'` {conum-guard-sh} # <2> ---- <1> Get the HTTP URL for invoking the action, the returned URL will have `.json` suffix to allow the WSK to set the right `Content-Type` and `Accept` headers <2> Some resources requires authentication, for those requests its required to add `Authorization` header with value as `$AUTH` [source,sh,subs=attributes+] ---- curl -k $WEB_URL ---- You can also access the url via browser using address held in variable $WEB_URL. [NOTE] ===== The following section shows some example requests and their expected responses **Without any request data** [source,json,subs=attributes+] ----- { "response": { "__ow_method": "get", "__ow_headers": { "x-forwarded-port": "443", "accept": "*/*", "forwarded": "for=192.168.64.1;host=openwhisk-faas.192.168.64.67.nip.io;proto=https", "user-agent": "curl/7.54.0", "x-forwarded-proto": "https", "host": "controller.faas.svc.cluster.local:8080", "x-forwarded-host": "openwhisk-faas.192.168.64.67.nip.io", "x-forwarded-for": "192.168.64.1" }, "__ow_path": "" } } ----- **With any JSON request data** [source,sh,subs=attributes+] ---- curl -k -X POST -H 'Content-Type: application/json' -d '{"name": "test"}' $WEB_URL.json ---- [source,json,subs=attributes+] ---- { "response": { "__ow_method": "post", "__ow_headers": { "x-forwarded-port": "443", "accept": "*/*", "forwarded": "for=192.168.64.1;host=openwhisk-faas.192.168.64.67.nip.io;proto=https", "user-agent": "curl/7.54.0", "x-forwarded-proto": "https", "host": "controller.faas.svc.cluster.local:8080", "content-type": "application/json", "x-forwarded-host": "openwhisk-faas.192.168.64.67.nip.io", "x-forwarded-for": "192.168.64.1" }, "__ow_path": "", "name": "test" } } ---- **With request data and an invalid content type** [source,sh,subs=attributes+] ---- curl -k -X POST -H 'Content-Type: application/something' -d '{"name": "test"}' $WEB_URL.json ---- Invoke via curl like above , with request data you will see the response like: [source,json,subs=attributes+] ---- { "response": { "__ow_method": "post", "__ow_headers": { "x-forwarded-port": "443", "accept": "*/*", "forwarded": "for=192.168.64.1;host=openwhisk-faas.192.168.64.67.nip.io;proto=https", "user-agent": "curl/7.54.0", "x-forwarded-proto": "https", "host": "controller.faas.svc.cluster.local:8080", "content-type": "application/something", "x-forwarded-host": "openwhisk-faas.192.168.64.67.nip.io", "x-forwarded-for": "192.168.64.1" }, "__ow_path": "", "__ow_body": "eyJuYW1lIjogInRlc3QifQ==" //<1> } } ---- <1> for unknown content-type the request body will be sent as base64 encoded string ===== === Chaining Actions Apache OpenWhisk allows chaining of actions which are called in the same sequence as they are defined. We will now create a simple sequence of actions which will split, convert to uppercase, and sort a comma separated string. All the three projects can be co-located in same directory for clarity and easy building: [source,sh,subs=attributes+] ----- cd .. mkdir -p sequence-demo cd sequence-demo wsk -i package create redhat-developers-demo {conum-guard-sh} <1> ----- <1> Create a new package to hold our actions, this gives a better clarity on which actions we add to our sequence. For more details refer to the {uri-openwhisk-docs-prefix}/packages.md[Packages] documentation. ==== Create Split Action This Action will receive a comma separated string as a parameter and return a array of Strings as a response. [source,sh,subs=attributes+] ---- cd $PROJECT_HOME/sequence-demo mvn archetype:generate \ -DarchetypeGroupId=org.apache.openwhisk.java \ -DarchetypeArtifactId=java-action-archetype \ -DarchetypeVersion=1.0-SNAPSHOT \ -DgroupId=com.example \ -DartifactId=splitter \ -Dversion=1.0-SNAPSHOT \ -DinteractiveMode=false ---- Update the FunctionApp class with this code: [source,java,subs=attributes+] ---- public static JsonObject main(JsonObject args) { JsonObject response = new JsonObject(); String text = null; if (args.has("text")) { text = args.getAsJsonPrimitive("text").getAsString(); } String[] results = new String[] { text }; if (text != null && text.indexOf(",") != -1) { results = text.split(","); } JsonArray splitStrings = new JsonArray(); for (String var : results) { splitStrings.add(var); } response.add("result", splitStrings); return response; } ---- Update the FunctionAppTest testFunction method with code: [source,java,subs=attributes+] ---- @Test public void testFunction() { JsonObject args = new JsonObject(); args.addProperty("text", "apple,orange,banana"); JsonObject response = FunctionApp.main(args); assertNotNull(response); JsonArray results = response.getAsJsonArray("result"); assertNotNull(results); assertEquals(3, results.size()); List actuals = new ArrayList<>(); results.forEach(j -> actuals.add(j.getAsString())); assertTrue(actuals.contains("apple")); assertTrue(actuals.contains("orange")); assertTrue(actuals.contains("banana")); } ---- ===== Build Splitter Action [source,sh,subs=attributes+] ---- cd splitter mvn clean package wsk -i action update redhat-developers-demo/splitter target/splitter.jar --main com.example.FunctionApp ---- ==== Create Uppercase Action This Action will take the array of Strings from previous step (Splitter Action) and convert the strings to upper case [source,sh,subs=attributes+] ---- cd .. mvn archetype:generate \ -DarchetypeGroupId=org.apache.openwhisk.java \ -DarchetypeArtifactId=java-action-archetype \ -DarchetypeVersion=1.0-SNAPSHOT \ -DgroupId=com.example \ -DartifactId=uppercase \ -Dversion=1.0-SNAPSHOT \ -DinteractiveMode=false ---- Update the FunctionApp class with this code: [source,java,subs=attributes+] ---- public static JsonObject main(JsonObject args) { JsonObject response = new JsonObject(); JsonArray upperArray = new JsonArray(); if (args.has("result")) { {conum-guard-java} // <1> args.getAsJsonArray("result").forEach(e -> upperArray.add(e.getAsString().toUpperCase())); } response.add("result", upperArray); return response; } ---- <1> The function expects the previous action in sequence to send the parameter with JSON attribute called `result` Update the FunctionAppTest testFunction method with code: [source,java,subs=attributes+] ---- @Test public void testFunction() { JsonObject args = new JsonObject(); JsonArray splitStrings = new JsonArray(); splitStrings.add("apple"); splitStrings.add("orange"); splitStrings.add("banana"); args.add("result", splitStrings); JsonObject response = FunctionApp.main(args); assertNotNull(response); JsonArray results = response.getAsJsonArray("result"); assertNotNull(results); assertEquals(3, results.size()); List actuals = new ArrayList<>(); results.forEach(j -> actuals.add(j.getAsString())); assertTrue(actuals.contains("APPLE")); assertTrue(actuals.contains("ORANGE")); assertTrue(actuals.contains("BANANA")); } ---- ===== Build Uppercase Action [source,sh,subs=attributes+] ---- cd uppercase mvn clean package wsk -i action update redhat-developers-demo/uppercase target/uppercase.jar --main com.example.FunctionApp ---- ==== Create Sort Action This Action will take the array of Strings from previous step (Upppercase Action) and sort them [source,sh,subs=attributes+] ---- cd .. mvn archetype:generate \ -DarchetypeGroupId=org.apache.openwhisk.java \ -DarchetypeArtifactId=java-action-archetype \ -DarchetypeVersion=1.0-SNAPSHOT \ -DgroupId=com.example \ -DartifactId=sorter \ -Dversion=1.0-SNAPSHOT \ -DinteractiveMode=false ---- Update the FunctionApp class with this code: [source,java,subs=attributes+] ---- public static JsonObject main(JsonObject args) { JsonObject response = new JsonObject(); List upperStrings = new ArrayList<>(); if (args.has("result")) { args.getAsJsonArray("result").forEach(e -> upperStrings.add(e.getAsString())); } JsonArray sortedArray = new JsonArray(); upperStrings.stream().sorted(Comparator.naturalOrder()).forEach(s -> sortedArray.add(s)); response.add("result", sortedArray); return response; } ---- Update the FunctionAppTest testFunction method with code: [source,java,subs=attributes+] ---- @Test public void testFunction() { JsonObject args = new JsonObject(); JsonArray splitStrings = new JsonArray(); splitStrings.add("APPLE"); splitStrings.add("ORANGE"); splitStrings.add("BANANA"); args.add("result", splitStrings); JsonObject response = FunctionApp.main(args); assertNotNull(response); JsonArray results = response.getAsJsonArray("result"); assertNotNull(results); assertEquals(3, results.size()); List actuals = new ArrayList<>(); results.forEach(j -> actuals.add(j.getAsString())); assertTrue(actuals.get(0).equals("APPLE")); assertTrue(actuals.get(1).equals("BANANA")); assertTrue(actuals.get(2).equals("ORANGE")); } ---- ===== Build Sorter Action [source,sh,subs=attributes+] ---- cd sorter mvn clean package wsk -i action update redhat-developers-demo/sorter target/sorter.jar --main com.example.FunctionApp ---- ==== Create an Action Sequence Having created all the three actions, lets now create OpenWhisk that calls all three function split,uppercase and sort in sequence. [source,sh,subs=attributes+] ---- cd .. wsk -i action update redhat-developers-demo/splitUpperAndSort --sequence redhat-developers-demo/splitter,redhat-developers-demo/uppercase,redhat-developers-demo/sorter ---- ====== Invoke and Verify [source,sh,subs=attributes+] ---- wsk -i action invoke redhat-developers-demo/splitUpperAndSort --param text "zebra,cat,antelope" --result ---- The above action invoke should result in response like: [source,sh,subs=attributes+] ---- { "result": [ "ANTELOPE", "CAT", "ZEBRA" ] } ---- === Advanced Check == Troubleshooting [[install-catalog]] === Reinstall default Catalog If you are on a low bandwidth sometimes the default catalog will not be populated, run the following commands to have them installed [source,sh,subs=attributes+] ---- #!/bin/bash oc delete job install-catalog <1> cat < ---- <1> Delete the old job <2> Run the install-catalog job again Now when you run `wsk -i action list` you should see output like: image::readme_images/ow_install_catalog.png[Install Catalog] [[tips-and-tricks]] == Tips and Tricks [TIP] ==== * If you are going to use a lot of `wsk` then its worth aliasing wsk with `alias wsk='wsk -i $@'` to avoid SSL errors and skip adding `-i` for every command. * For detailed JSON output form `wsk` commands prefix `-v`. This is a great command option for troubleshooting. * Its safe to use `wsk -i update [resource]` when creating OpenWhisk resources like **Actions**, **Packages** etc., as this command will act like `create` for new resources and `update` for existing resources. * `wsk -i [resource command ] --summary` provides detailed information about a specific resource e.g. wsk -i action get foo --summary * `wsk -i activation poll`, a very useful command when we want to debug some error or see the exception stack traces during a function execution. The simple example of this could be that we star this command on one terminal and fire the action on another to see poll window showing exceptions/errors/stacktraces if any during execution. ==== [[references]] == References * {uri-openwhisk-openshift}[Apache OpenWhisk on OpenShift] * {uri-openwhisk-docs-prefix}/actions.md[OpenWhisk Actions] * {uri-openwhisk-docs-prefix}/cli.md[Setup OpenWhisk CLI] * {uri-openwhisk-docs-prefix}/packages.md[Packages] * {uri-openwhisk-docs-prefix}/webactions.md[Web Action]