Arkade Install Kafka fails

remiconnesson commented 1 year ago

Expected Behaviour

arkade install kafka

Current Behaviour

VALUES values.yaml

Command: /home/remi/.arkade/bin/helm [upgrade --install cp-helm-charts confluentinc/cp-helm-charts --namespace  --values /tmp/charts/cp-helm-charts/values.yaml --set cp-zookeeper.servers=1 --set cp-kafka.enabled=true --set cp-schema-registry.enabled=false --set cp-kafka-rest.enabled=false --set cp-kafka-connect.enabled=false --set cp-ksql-server.enabled=false --set cp-control-center.enabled=false --set cp-zookeeper.enabled=true --set cp-kafka.brokers=1]

Release "cp-helm-charts" does not exist. Installing it now.

Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "cp-helm-charts-cp-zookeeper-pdb" namespace: "" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
ensure CRDs are installed first

Error: exit code 1, stderr: Error: unable to build kubernetes objects from release manifest: resource mapping not found for name: "cp-helm-charts-cp-zookeeper-pdb" namespace: "" from "": no matches for kind "PodDisruptionBudget" in version "policy/v1beta1"
ensure CRDs are installed first

Are you a GitHub Sponsor yet (Yes/No?)

[ ] Yes
[X] No

Possible Solution

Steps to Reproduce (for bugs)

arkade get k3sup
export PATH=$PATH:$HOME/.arkade/bin/ in .zshrc, then source
k3sup install --local --k3s-extra-args "--write-kubeconfig-mode=644"
(wait)
arkade install kafka
(above results in an error)

Context

I'm using arkade because it's the only tool I understand yet, I'm not versed in helm yet. So basically my only shot to install kafka on k3s

This being said, not really affecting me I'm using it for a toy project

I'm opening this issue fyi because every install I did with arkade worked out of the box, it's the first time it doesn't. Beautiful tool.

Your Environment

What Kubernetes distribution are you using?

K3S on ubuntu 22 single node

WARNING: This version information is deprecated and will be replaced with the output from kubectl version --short.  Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.5+k3s2", GitCommit:"de654222cb2c0c21776c32c26505fb684b246a1b", GitTreeState:"clean", BuildDate:"2023-01-11T21:23:33Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.5+k3s2", GitCommit:"de654222cb2c0c21776c32c26505fb684b246a1b", GitTreeState:"clean", BuildDate:"2023-01-11T21:23:33Z", GoVersion:"go1.19.4", Compiler:"gc", Platform:"linux/amd64"}

Operating System and version (e.g. Linux, Windows, MacOS):

➜  ~ uname -a
Linux <redacted> 5.15.0-46-generic #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
➜  ~ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

What arkade version is this?


Version: 0.8.59
Git Commit: bdb3869d5cf87ccc0d1d2ec0c9ecffe113e31ce6```

alexellis commented 1 year ago

Thanks for the feedback.

@Jasstkn is this something you could try out and look into?

Jasstkn commented 1 year ago

Thanks for the feedback.

@Jasstkn is this something you could try out and look into?

I'll take a look later today.

Jasstkn commented 1 year ago

@remiconnesson @alexellis the reason is the following: kafka's charts aren't well maintained. There is a lot of issues that are currently opened. One of them addresses issue with PDB that you faced: https://github.com/confluentinc/cp-helm-charts/pull/629.

It means that arkade install kafka won't work on the Kubernetes v1.21+

remiconnesson commented 1 year ago

Thank you very much for answering that fast

In the meantime, For anyone reading this wanting to install kafka on k3s I'm currently looking into this solution:

https://github.com/IBM/kraft-mode-kafka-on-kubernetes (to create the docker image)
https://learnk8s.io/kafka-ha-kubernetes (to show how to use it)

alexellis commented 1 year ago

@remiconnesson Confluent provide free Kafka for development. Have you looked into that? We use it for testing the OpenFaaS Pro Kafka Connector

alexellis commented 1 year ago

Any other suggestions for charts @Jasstkn ?

Jasstkn commented 1 year ago

Any other suggestions for charts @Jasstkn ?

https://github.com/bitnami/charts/tree/main/bitnami/kafka?

mrwormhole commented 1 year ago

looks like it is using "policy/v1beta1", it is deprecated in v1.21 and completely removed in v1.25 in favor of "policy/v1", mostly helm chart should use this in PodSecurityPolicy or PodDisruptionBudget

i would use bitnami charts instead of confluentinc ones, they are more maintained @Jasstkn