Replace Bitnami with Strimzi Kafka

nasark commented 1 year ago

install Strimzi by creating a subscription from the community-operators OLM CatalogSource
move miq-components/kafka.go to its own package miq-components/kafka/kafka.go, this will allow the CRs to be used elsewhere (i.e. downstream)

Depends on:

TODO:

[x] Since Strimzi requires SSL, need to mount the ca.crt in orchestrator/workers regardless of whether internal-certificate-secret is present or not
[x] set node affinity in Kafka CR
[x] drop bitnami references in CR (marked as deprecated in CR)
[x] topic management to be handled by orchestrator? (not needed for now)

@miq-bot assign @bdunne @miq-bot add_reviewer @Fryguy @miq-bot add_label enhancement

nasark commented 11 months ago

@bdunne @Fryguy Please review

bdunne commented 11 months ago

Close/reopen to kick tests

nasark commented 11 months ago

We'll need to come up with an upgrade plan at some point. Do you just change the channel name?

Yes, the channel name is changed to a release available here https://operatorhub.io/operator/strimzi-kafka-operator

nasark commented 11 months ago

Applied the following feedback:

Limit/requests with cr.Spec.EnforceWorkerResourceConstraints, move to mutate function
Use addLabel function for labels (dependent PR https://github.com/ManageIQ/manageiq-pods/pull/1020)
Productize names
Use subscription api instead of unstructured objects

nasark commented 11 months ago

Applied the following:

reworked renewKafkaCASecret() and relevant code to use caGen stored in last-known-revision annotation
- also a bug fix for updating unstructured kafka objects (explained below)
Check if openshift is being used by looking for openshift-marketplace namespace and relevant catalog source
Create operatorgroup in order to allow creation of strimzi subscription from catalogsource

nasark commented 10 months ago

@miq-bot add_label quinteros/yes?

miq-bot commented 10 months ago

Checked commits https://github.com/nasark/manageiq-pods/compare/80790f673c4909a5c62562fa874402060aee9175~...a1fb2fb2a913cb2737a32bf74722a869c53acc3f with ruby 2.7.8, rubocop 1.56.3, haml-lint 0.51.0, and yamllint 1 file checked, 1 offense detected

**

[ ] :bomb: :boom: :fire: :fire_engine: - Linter/Yaml - missing config files

Fryguy commented 10 months ago

Backported to quinteros in commit 22aa97ad00f6db6097eb1e65300aff2a894fddd5.

commit 22aa97ad00f6db6097eb1e65300aff2a894fddd5
Author: Brandon Dunne <brandondunne@hotmail.com>
Date:   Thu Jan 4 17:18:11 2024 -0500

    Merge pull request #1005 from nasark/strimzi_kafka

    Replace Bitnami with Strimzi Kafka

    (cherry picked from commit 0ccfda19e605fa147218ce6ee3881c2f1934ca83)

ManageIQ / manageiq-pods

Replace Bitnami with Strimzi Kafka #1005