Closed sehz closed 2 years ago
This is not verified to working with Docker. This have to be done with cluster installer reworking
closing this for now unless someone requested support for Docker for Mac
Hyperkit driver works for minikube here: https://www.fluvio.io/docs/get-started/mac/. closing it
hyperkit doesn't work on M1 so need another workaround
Latest K3d removed --no-hostip -
option so need solution quickly
Key issue is that Docker doesn't forward ports from minikube containers. This can be work around in the kind by creating using this config:
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
# port forward 80 on the host to 80 on this node
extraPortMappings:
- containerPort: 30007
hostPort: 30007
# optional: set the bind address on the host
# 0.0.0.0 is the current default
listenAddress: "127.0.0.1"
# optional: set the protocol to one of TCP, UDP, SCTP.
# TCP is the default
protocol: TCP
Assuming fluvio adds node port mapping for public services
apiVersion: v1
kind: Service
metadata:
name: fluvio-sc-public
annotations:
{{- toYaml .Values.loadBalancer.serviceAnnotations | nindent 4 }}
spec:
type: {{ .Values.service.type }}
selector:
app: fluvio-sc
{{ if .Values.service.externalTrafficPolicy }}
externalTrafficPolicy: .Values.service.externalTrafficPolicy
{{ end }}
{{ if .Values.service.externalName }}
externalName: .Values.service.externalName
{{ end }}
ports:
- protocol: TCP
port: 9003
targetPort: 9003
nodePort: 30007
The minikube command for port forwarding doesn't work on current version and it is not customizable as kind:
minikube start --driver=docker --extra-config=apiserver.service-node-port-range=32760-32767 --ports=127.0.0.1:32760-32767:32760-32767
All container system (including docker) is using QEMU HVF. Currently available container runtime are:
Main issue with Docker is that it can't be used in Github Action CI due to license restriction. Given this, we should move toward standard container runtime to simplify our development and testing.
Criteria for selecting runtime are:
Podman doesn't work with Kind yet:
KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Cgroup controller detection is not implemented for Podman. If you see cgroup-related errors, you might need to set systemd property "Delegate=yes", see https://kind.sigs.k8s.io/docs/user/rootless/
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.21.1) 🖼
✗ Preparing nodes 📦
ERROR: failed to create cluster: podman run error: command "podman run --hostname kind-control-plane --name kind-control-plane --label io.x-k8s.kind.role=control-plane --privileged --tmpfs /tmp --tmpfs /run --volume 56dcb311adfa416309a7348d0fbe394ba80ce4267330b95257f6574818b264a3:/var:suid,exec,dev --volume /lib/modules:/lib/modules:ro --detach --tty --net kind --label io.x-k8s.kind.cluster=kind -e container=podman --publish=127.0.0.1:50715:6443/tcp -e KUBECONFIG=/etc/kubernetes/admin.conf kindest/node@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6" failed with error: exit status 126
Command Output: Error: error configuring network namespace for container 47704ccffa0ec65232dacc224c606d900f71d08e4e51232bfc43733bab2b3f90: error adding pod kind-control-plane_kind-control-plane to CNI network "kind": failed to list chains: running [/usr/sbin/ip6tables -t nat -S --wait]: exit status 3: modprobe: ERROR: could not insert 'ip6_tables': Operation not permitted
ip6tables v1.8.7 (legacy): can't initialize ip6tables table `nat': Table does not exist (do you need to insmod?)
Perhaps ip6tables or your kernel needs to be upgraded.
Rancher desktop seems to provide comprehensive support for Kubernetes for both Mac, Windows and Linux. Currently Apple Silicon is in-progress. Rancher desktop is built on of Lima
For VM, Lima seems to be more flexible since it supports many distribution while Podman seems to support primary fedora. Overall, ecosystem is still unstable for making judgement although Lima seems to be most promising.
Relevant link: https://github.com/kubernetes/minikube/issues/11193.
Docker does not provide way for host access container port directly.