Closed sanket1211996 closed 2 years ago
Dev Notes:
Delete Stuck Namespace: In one terminal:
kubectl proxy In another terminal:
kubectl get ns delete-me -o json | \ jq '.spec.finalizers=[]' | \ curl -X PUT http://localhost:8001/api/v1/namespaces/delete-me/finalize -H "Content-Type: application/json" --data @-
Dev Notes: Dev Notes Instance Setup
https://microk8s.io/?_ga=2.251306115.466575059.1648492247-887695204.1648492247
https://stackoverflow.com/questions/58654118/pulling-local-repository-docker-image-from-kubernetes
sudo microk8s enable metallb
Dev Notes Kafka Kube Install: https://docs.bitnami.com/tutorials/deploy-scalable-kafka-zookeeper-cluster-kubernetes/
helm repo add bitnami https://charts.bitnami.com/bitnami
helm install -n space-dev zookeeper bitnami/zookeeper --set replicaCount=1 --set auth.enabled=false --set allowAnonymousLogin=true
helm install -n space-dev kafka bitnami/kafka --set zookeeper.enabled=false --set replicaCount=1 --set externalZookeeper.servers=zookeeper.space-dev.svc.cluster.local
helm upgrade -n space-dev zookeeper bitnami/zookeeper --set replicaCount=1 --set auth.enabled=false --set allowAnonymousLogin=true
helm upgrade -n space-dev kafka bitnami/kafka --set zookeeper.enabled=false --set replicaCount=1 --set externalZookeeper.servers=zookeeper.space-dev.svc.cluster.local
Zookeeper help instructions: Please be patient while the chart is being deployed
ZooKeeper can be accessed via port 2181 on the following DNS name from within your cluster:
zookeeper.space-dev.svc.cluster.local
To connect to your ZooKeeper server run the following commands:
export POD_NAME=$(kubectl get pods --namespace space-dev -l "app.kubernetes.io/name=zookeeper,app.kubernetes.io/instance=zookeeper,app.kubernetes.io/component=zookeeper" -o jsonpath="{.items[0].metadata.name}")
kubectl exec -it $POD_NAME -- zkCli.sh
To connect to your ZooKeeper server from outside the cluster execute the following commands:
kubectl port-forward --namespace space-dev svc/zookeeper 2181: &
zkCli.sh 127.0.0.1:2181
Kaafka Help Instructions: Please be patient while the chart is being deployed
Kafka can be accessed by consumers via port 9092 on the following DNS name from within your cluster:
kafka.space-dev.svc.cluster.local
Each Kafka broker can be accessed by producers via port 9092 on the following DNS name(s) from within your cluster:
kafka-0.kafka-headless.space-dev.svc.cluster.local:9092
To create a pod that you can use as a Kafka client run the following commands:
kubectl run kafka-client --restart='Never' --image docker.io/bitnami/kafka:3.1.0-debian-10-r60 --namespace space-dev --command -- sleep infinity
kubectl exec --tty -i kafka-client --namespace space-dev -- bash
PRODUCER:
kafka-console-producer.sh --broker-list kafka-0.kafka-headless.space-dev.svc.cluster.local:9092 --topic test
CONSUMER:
kafka-console-consumer.sh --bootstrap-server kafka.space-dev.svc.cluster.local:9092 --topic test --from-beginning
Kafka Topic : kafka-topics.sh --create --create --topic nexrad_incoming --bootstrap-server kafka-0.kafka-headless.space-dev.svc.cluster.local:9092
kafka-topics.sh --create --create --topic nexrad_outgoing --bootstrap-server kafka-0.kafka-headless.space-dev.svc.cluster.local:9092
kafka-topics.sh --create --create --topic merra_incoming --bootstrap-server kafka-0.kafka-headless.space-dev.svc.cluster.local:9092
kafka-topics.sh --create --create --topic merra_outgoing --bootstrap-server kafka-0.kafka-headless.space-dev.svc.cluster.local:9092
kafka-topics.sh --list --bootstrap-server kafka-0.kafka-headless.space-dev.svc.cluster.local:9092
Kafka Manager Setup:
helm install stable/kafka-manager zkHosts zookeeper.space-dev.svc.cluster.local
helm upgrade -n space-dev zookeeper bitnami/zookeeper --set replicaCount=2 --set auth.enabled=false --set allowAnonymousLogin=true
helm upgrade -n space-dev kafka bitnami/kafka --set zookeeper.enabled=false --set replicaCount=2 --set externalZookeeper.servers=zookeeper.space-dev.svc.cluster.local
kafka-0.kafka-headless.space-dev.svc.cluster.local:9092 kafka-1.kafka-headless.space-dev.svc.cluster.local:9092