Add Egeria operator

[planetf1]

Operators are software extensions to Kubernetes that make use of custom resources to manage applications and their components.

For example we could have an 'egeria' custom resource definition (or multiple) that define what an egeria deployment should look like. An egeria operator would be constantly comparing this definition with reality, and where this differs will 'make it so' to quote a famous captain.

I have a sample, simplistic egeria operator which I'll import into the egeria code base & validate the build process/logistics, before returning later to review the design aspects of what should be in the CRD

[planetf1]

Note that the operator sdk move to 1.0 has introduced quite a number of changes to the templating of the operator & how it';s built mostly to be consistent with kubebuilder... Old operators are already being referred to as 'legacy'

more detail at https://github.com/operator-framework/operator-sdk/issues/3327. There's also a discussion at https://groups.google.com/g/operator-framework/c/ZXxnSVi_fWU. A migration guide is at https://sdk.operatorframework.io/docs/golang/project_migration_guide/

A pre-release was reported as being within weeks 2 weeks ago so should be imminent

Moving to this model for the sdk seems sensible for the first prototype in the egeria code base. The runtime version supported is from 1.13 and above (commerical products are in the 1.16-1.18 area) so no undue concerns about dependencies.

[planetf1]

Operator SDK

What is an operator?

• Packing of a Kubernetes applications
• Operator is an applications specific controler that can create, configure & manage instances
• Uses application specific knowledge to manage lifecycle
• As a controller, uses a control loop constantly reconciling observed state ('watches') to desired state
• Uses a Custom Resource Definition (CRD) to define config available to users & set desired starter
• API can do anything - scaling, deploy, backup
• Especially well suited to stateful applications where standard k8s control needs extending

operator tooling

• Can use basic k8s apis
• operator-sdk (redhat) - sdk supporting helm, ansible, go based operators
• Metacontroller - started at google
• Kopf (Zalando)
• kubebuilder
• side 8's k8s-operator - geared around python, though can use other languages for the operator including bash

Of these operator-sdk seems to have the most significant footprint. v1 has just been released, with many 0.x functions deprecated, so starting off with v1 seems prudent. v1 also works much better with kubebuilder

Note much of the toolkit's functions are around code generation - so does not handle the full loop, and upgrades in standard patterns are somewhat obscured :-(

Whilst helm charts can be wrapped in an operator, as can ansible scripts, go seems the most logical approach given k8s itself is written in go, and a majority of operators appear to be

(Aside - will also integrate with the operator lifecycle manager)

-> use operator-sdk, go

Go operators

Probably the most common. k8s itself is developed in go. The most flexible, but also does require go programming skills - though this is common for k8s.

Helm operators

Basically maps the values.yaml file to a CRD Quick start, but evolution will require working with helm also - and helm isn't good at lifecycle management (though there is still a role for helm with operators IMO - and that is in packaging a full application stack for demos)

Ansible operators

• sdk creates some simple go code for binding to Ansible
• define events to watch (including CRD)
• run ansible playbooks in response to events
• Since sdk creates the go template this can also be modified - resulting in a hybrid operator
• Still requires go tooling and potential minor code mods in go
• May be possible to share some resources with non-k8s environments

Questions so far seem to conclude go is best, but significant learning curve

Egeria build

• Currently we build with maven. Gradle also evolving.
• There are decent looking go plugins for both, especially Gradle
• operator-sdk is 'make' and script focussed (makefiles created by tooling)
• some steps templated by the sdk are closer to deployment than building an artifacts
• combination of new language, build tool, toolkit, os dependencies, user role, and new-ness (v1 sdk) suggests keeping in a new odpi egeria may be easiest

-> use a new clean repo

Operator capabilities

• I built a sample operator which can just launch 1 or more egeria instances (and scale dynamically) - just to get some hands on time.

  scenarios

• Install (baseline!)
• config changes
• Upgrade (one egeria release to next?)
• backup (what does this mean)
• failure recovery
• auto scaling - future?
• adding platforms - y
• adding servers?
• configuration of event bus etc

-> based on these we define a CRD which defines the Egeria object & it's attributes -- we can create multiple objects representing different things if we wish, like a cohort

Scope - focus on core egeria. Kafka, ldap etc should be pointed to externally. A helm chart could deploy these additional components for a demo?

[planetf1]

As of 034ae14f12257349a797a18f22cb2287524fc4f1 the operator is now able to launch instances of egeria (2.2-SNAPSHOT).

This is a first step - an operator can be built, deployed, and run egeria.

As such I shall make finer grained issues for followon activities