CORTX-34416: CORTX Deployment is failing in Kafka installation

shailesh-vaidya commented 1 year ago

Description

CORTX Deployment is failing in Kafka installation. The error reported is as below,

11:34:05  ######################################################
11:34:05  # Deploy CORTX                                        
11:34:05  ######################################################
11:34:08  Getting updates for unmanaged Helm repositories...
11:34:08  ...Successfully got an update from the "https://helm.releases.hashicorp.com/" chart repository
11:34:08  ...Successfully got an update from the "https://charts.bitnami.com/bitnami" chart repository
11:34:09  Error: INSTALLATION FAILED: can't get a valid version for repositories kafka. Try changing the version constraint in Chart.yaml

Type of change

[x] Bug fix (non-breaking change that fixes an issue)
[ ] New feature (non-breaking change that adds new functionality)
[ ] Breaking change (bug fix or new feature that breaks existing functionality)
[ ] Third-party dependency update
[ ] Documentation additions or improvements
[ ] Code quality improvements to existing code or test additions/updates

Applicable issues

This change fixes an issue: #
This change is related to an issue: #

CORTX image version requirements

This change requires the following images:

cortx-data:<version>
cortx-rgw:<version>
cortx-control:<version>

How was this tested?

Validated deployment with Jenkins pipeline execution. Refer - https://eos-jenkins.colo.seagate.com/job/Cortx-Automation/job/RGW/job/setup-cortx-rgw-cluster/13085/console

Additional information

On investigation found that the Kafka version mentioned in Chart.yaml is not available in bitnami index

[root@ssc-vm-rhev4-2985 k8_cortx_cloud]# helm install my-release bitnami/kafka --version 16.2.7
Error: INSTALLATION FAILED: chart "kafka" matching 16.2.7 not found in bitnami index. (try 'helm repo update'): no chart version found for kafka-16.2.7

Updated the nearest version in Chart.yaml and validated deployment through Jenkins pipeline execution.

Checklist

[x] The change is tested and works locally.
[] New or changed settings in the solution YAML are documented clearly in the README.md file.
[x] All commits are signed off and are in agreement with the CORTX Community DCO and CLA policy.

If this change requires newer CORTX or third party image versions:

[ ] The image fields in solution.example.yaml have been updated to use the required versions.
[ ] The appVersion field of the Helm chart has been updated to use the new CORTX version.

If this change addresses a CORTX Jira issue:

[x] The title of the PR starts with the issue ID (e.g. CORTX-XXXXX:)

keithpine commented 1 year ago

To avoid this problem in the future, it would be prudent to create Seagate-owned mirrors of the Helm repos and reference those instead.

To update the README file, install helm-docs and run the same command, as seen in the GH workflow. Just push a new commit to this branch and the PR tests will pass.

https://github.com/Seagate/cortx-k8s/blob/601ddb63017cd6dc13a9ad866159c81f1cfb2650/.github/workflows/lint.yaml#L85-L92

Or just manually replace the version number with 17.2.3, which is what the docs command will do for you.

BTW, solution.example.yaml references the kafka and zookeeper images, and this overrides the default Chart value:

https://github.com/Seagate/cortx-k8s/blob/601ddb63017cd6dc13a9ad866159c81f1cfb2650/k8_cortx_cloud/solution.example.yaml#L20-L21

As seen, this is using 3.0.0 still. Chart v16.2.7 specifies kafka image 3.1.0-debian-10-r85, so we were already out of date, and 17.2.3 uses 3.2.0-debian-10-r4, so a difference of 3.0 vs 3.2. I don't see anything obvious in the Charts that would cause any problem here, just making note of it. I assume you already verified 3.0.0 is still working fine.

shailesh-vaidya commented 1 year ago

@osowski, @keithpine - Thanks for the input. Added fix and Lint / Lint and Test Helm Charts is passing now. Lint / Shellcheck is still failing. Can you please advice.

keithpine commented 1 year ago

The shellcheck action is using the newest version of shellcheck by default (0.9.0 vs 0.8.0) which seems to flag more problems. You can update this PR and pin the action to version 0.8.0 in file .github/workflows/lint.yaml (or fix the problems).

      - name: Run Shellcheck
        uses: Seagate/action-shellcheck@203a3fd018dfe73f8ae7e3aa8da2c149a5f41c33
        with:
          version: v0.8.0

shailesh-vaidya commented 1 year ago

@keithpine - Thanks for the suggestion. @osowski - All checks are passing now. Please review the pull request changes.

Seagate / cortx-k8s