Configuring calico-node on non cluster hosts using the docker image does not properly register nodes into ETCD

luizrojo commented 6 months ago

Expected Behavior

Configuring calico-node on non cluster hosts should register the host endpoints into ETCD and on the Calico Cluster.

Current Behavior

Running calico-node does not add the host endpoints, not properly setting things up.

Possible Solution

Looks like the documentation is incorrect/missing some steps to set things up.

Steps to Reproduce (for bugs)

Set up a ETCD Cluster

Configure calico-node on a non-cluster host and start it

/usr/bin/docker run --net=host --privileged \
--name=calico-node \
-e CALICO_NODENAME=host1 \
-e IP=10.140.140.189 \
-e IP6= \
-e CALICO_NETWORKING_BACKEND=none \
-e AS=1234 \
-e NO_DEFAULT_POOLS=true \
-e DATASTORE_TYPE=etcdv3 -e ETCD_ENDPOINTS=https://etcd1:2379,https://etcd2:2379,https://etcd3:2379 \
-e ETCD_CA_CERT_FILE=/etc/pki/tls/certs/ca-bundle.crt \
-e ETCD_CERT_FILE=/etc/pki/acme/certs/host1/cert.pem \
-e ETCD_KEY_FILE=/etc/pki/acme/private/host1/privkey.pem \
-e KUBECONFIG= \
-v /var/log/calico:/var/log/calico \
-v /var/lib/calico:/var/lib/calico \
-v /var/run/calico:/var/run/calico \
-v /run/docker/plugins:/run/docker/plugins \
-v /lib/modules:/lib/modules \
-v /etc/pki:/pki \
-v /etc/pki:/etc/pki \
internalregistry/calico/node:v3.27.2 /bin/calico-node -felix

Check if host has been registered on ETCD/Calico

# calicoctl get nodes
NAME
# calicoctl get hostendpoints
NAME

Running the container with the -startup parameter seems to register the host as a node, but it is not added as a hostendpoint (which I think is expected), but it also initializes the default IPAM

Context

I am running a POC to set up calico as a FW management on VM/Baremetal server outside the K8s clusters we have in place

Your Environment

Calico version: v3.27.2
Orchestrator version (e.g. kubernetes, mesos, rkt): No oschestrator in place, things are running on virtual machines
Operating System and version: Oracle Linux 9.3

luizrojo commented 6 months ago

Thread on slack that originated this issue

fasaxc commented 6 months ago

The doc just looks wrong to me; it runs calico-node -felix which will run only felix, not the startup scripts that would initialise the datastore as it suggests.

Best workaround is probably to run Step 5 here: https://docs.tigera.io/calico/latest/getting-started/bare-metal/installation/binary#step-5-initialize-the-datastore I.e. just write the Node resource to the datastore yourself. I'm not sure if the startup scripts will work outside of Kubernetes.

projectcalico / calico