Deployment Components

[dragndropzek]

Hello,

I would have posted this on community slack, but I was having issues getting access to it.

I'm trying to understand all components that OpenWhisk requires as part of the deployment, but having trouble piecing it all together since documentation seems to be not all in sync within the repo and OpenWhisk website.

I'm reading docs / OpenWhisk website and I'm seeing that Whisk is built on top of below components:

NGINX (SSL Termination) Controller CouchDB / IBM Cloudant Kafka Invokers (Docker) Above makes sense from what I'm reading how Whisk works.

However, while reading some Ansible documentation, I noticed that there are also mentions of below components that don't seem to be part of official docs or docs on official OpenWhisk website.

MongoDB Etcd Redis Elastic Search Is there documentation I can read on when each component is required and if certain components are optional?

Lastly, does OpenWhisk support OpenSearch if Elastic is supported?

Thanks,

[kcexn]

I've been having issues with the Community Slack too.

The OpenWhisk architecture has as key components:

1. The Controller
2. The Invoker
3. A Database
4. A Message Queue. (5). Optionally an additional Scheduler.

Most of the features in OpenWhisk are implemented using Service Provider Interfaces (SPI), and so many components can be specialized towards a specific implementation depending on your particular operational environment.

The `default' (or simplest) deployment, consists of:

1. NGINX (load balancer)
2. Controllers. (Authentication, Load Balancing, Scheduling Services)
3. CouchDB (Database service)
4. Kafka (Message Queue)
5. Invokers (Container Management Services)
6. API Gateway (Utilities for Exposing Web Actions)

In response to your specific points,

MongoDB can be used as an alternative to CouchDB by configuring it in the relevant properties files in the Controller and Invokers. Template Redis is used by the API Gateway. Documentation Elastic Search can be used as an alternative action log store. Action Log Store Documentation

and Etcd is used by the new Scheduler component, which is most useful if you are deploying OpenWhisk on top of a container orchestration engine like Kubernetes. A little bit of the scheduler documentation can be found here.

I agree with your general remark that some of the documentation is a bit out of date and disorganized though...

[style95]

There are two versions of OpenWhisk at the moment.

One is based on the ShardingPoolBalancer and the other is the new scheduler(FPCPoolBalancer). We plan to release the next version of OpenWhisk that uses the new scheduler by default. (For now, the ShardingPoolBalancer is enabled by default.)

As @kcexn described above, the new scheduler requires as the name stands for, one more component "scheduler". And this scheduler component requires ETCD.

Other than ETCD, other components are orthogonal to core components. You can select ArtifactStore which stores actions and ActivationStore which stores activations. For ArtifactStore, you can choose one of CouchDB and MongoDB; for ActivationStore, you can choose one of CouchDB, MongoDB, and ElasticSearch. These stores can be used no matter which version of OpenWhisk you use.

Regarding the documentation, these days, we are short-handed, so solicit help.

[kcexn]

@style95 with regards to documentation.

The biggest problem to me seems like organisation. Most things seem to be there, but not easily found.

I think maybe if there was a hierarchy or a better sense of organisation around how the documentation should work, maintaining it would be much easier.

E.g. Maybe organise everything in a directory tree. With subdirectories for Development, Deployment, Architecture etc.

Might be worth considering as an enhancement proposal?

I think once the documentation has a structure, it will be easier for members of the community to help keep it up to date. As it stands, it's often unclear whether documentation is missing, incomplete, out of date, or I haven't looked hard enough.

I don't even know where to even begin if I wanted to help improve the documentation.

[style95]

@kcexn Thank you for the suggestion. That makes sense to me.

I will keep that in mind when updating documentation. I think we need to redefine and update many things. The Slack invitation is also one of them.

For now, you guys can request invitations via the dev list as mentioned here. https://openwhisk.apache.org/community.html

[dragndropzek]

Thanks for all the info. I'll see how I can contribute to the project.

I would recommend for Github documentation to be reflected on main OpenWhisk site https://openwhisk.apache.org/documentation.html in order to keep things consistent.