OLM Manifests for release 24.11

[razvan]

Operators

TODO

• update the image refs after the final release

Please follow this list in order at least until the zookeeper-operator before parallelizing.	Operator	Maintainer	Branch	Kuttl Tests	Cert Pipeline	Notes
commons	@razvan	stackable-commons-24.11.0	✅
secret	@razvan	stackable-secret-24.11.0	✅
listener	@razvan	stackable-listener-24.11.0	✅
zookeeper	@razvan	stackable-zookeeper-24.11.0	✅
hdfs	@razvan	stackable-hdfs-24.11.0	✅
hive	@adwk67	stackable-hive-24.11.0	✅
hbase	@adwk67	stackable-hbase-24.11.0	✅
opa	@adwk67	stackable-opa-24.11.0	✅
druid	@adwk67	stackable-druid-24.11.0	✅
kafka	@adwk67	stackable-kafka-24.11.0	✅
nifi	@adwk67	stackable-nifi-24.11.0	✅
spark	@adwk67	stackable-spark-24.11.0	✅
superset	@razvan	stackable-superset-24.11.0	✅
airflow	@razvan	stackable-airflow-24.11.0	✅
trino	@adwk67	stackable-trino-24.11.0	✅

Description

Part of: https://github.com/stackabletech/issues/issues/647

This is a list of steps you can follow to generate, install and test Operator Lifecycle Management (OLM) manifests for the Stackable Data Platform.

The manigest generation is fully automated for all operators except two: the secret and the listener operator. These two required manual intervention.

OLM manifests can be generated at any time and for any version of the operators (including 0.0.0-dev) but it's specially needed after a platform release, to certify the new operator versions.

To generate manifests for a released version of the SDP, ensure that you checkout the appropriate branch of the operator repository and the openshift operator repository.

Pre-requisites

In addition to the usual development tools like git and Python you need to clone the following repositories, idealy in the same location (this guide assumes $HOME/repo/stackable):

• https://github.com/stackabletech/stackable-utils - contains utility scripts that you'll use below
• https://github.com/stackabletech/openshift-certified-operators - a fork used to submit SDP operators for certification
• Operator repositories that you intend to use like:
  • https://github.com/stackabletech/airflow-operator/
  • https://github.com/stackabletech/druid-operator/
  • https://github.com/stackabletech/commons-operator/
  • ... and so on

This issue assumes you are generating OLM manifests for a released SDP version with the intention of certifying it. This means:

• [ ] All operator repositories have the required release branch. These releases have been tested and are generally available.
• [ ] All container images have been published to quay.io

Secret and listener operator

Generate manifests

• [ ] Update the supported OpenShift version range in the function generate_metadata()

• [ ] Create release branch (from main) in the openshift operator repository

  cd $HOME/repo/stackable/openshift-certified-operators git switch -c stackable-secret-24.3.0

• [ ] Generate manifests

  ./olm/build-manifests.sh -r 24.3.0 \ -c $HOME/repo/stackable/openshift-certified-operators \ -o $HOME/repo/stackable/secret-operator

Options:

• -r 24.3.0 is the SDP release

• -c $HOME/repo/stackable/openshift-certified-operators the location of the openshift operator repository

• -o $HOME/repo/stackable/secret-operator the location of the secret op

• [ ] Copy the cluster service version file from the previous version.

• [ ] Copy the operator manifests file fro the previous version. For secret and listener

• [ ] Replace the contents of the deployment, and cluster role with the template.spec and rules from the newly generated files.

• [ ] Remove the unused generated files : service account, operator cluster role (not the product cluster role), role binding, deployment.

• [ ] Remove all Helm labels in all remaining files (including all labels from the cluster role).

• [ ] Check or update the metadata/dependencies.yaml

• [ ] Update image tags and hashes with quay.io versions

• [ ] Commit & push your changes

Install manifests

• [ ] Ensure your K8S configuration points to a an OpenShift (or OKD) instance

• [ ] Install the operator from the manifests generated in the previous step

  /olm/build-bundles.sh -r 24.3.0 -o secret -c ~/repo/stackable/openshift-certified-operators -d

Options:

• -r 24.3.0 version of the operator to install
• -o secret name of the operator to install
• -c ~/repo/stackable/openshift-certified-operators location of the openshift operator repository
• -d deploy to the cluster

All other operators

These operators shouldn't require any manual editing of the manifests but a visual check is recommended.

The steps are illustrated only once for the ZooKeeper operator but a list of all operators to work on is added below.

Generate manifests

• [ ] Create release branch (from main) in the openshift operator repository

  cd $HOME/repo/stackable/openshift-certified-operators git switch -c stackable-zookeeper-24.3.0

• [ ] Ensure appropriate branch (release-24.3) is checkout out in the ZooKeeper operator

• [ ] Generate manifests

  ./olm/build-manifests.py \ --openshift-versions 'v4.11-v4.15' \ --release 24.3.0 \ --repo-operator ~/repo/stackable/zookeeper-operator

See ./olm/build-manifests.py --help for possible options.

Install manifests

./olm/build-bundles.sh \
-r 24.3.0 \
-o zookeeper \
-c ~/repo/stackable/openshift-certified-operators \
-d

Test the operator

Run the openshit integration test suite for the operator

 cd ~/repo/stackable/zookeeper-operator
 /scripts/run-tests --test-suite openshift --skip-release

We use --skip-release because the operator has already been installed previously.