odpi / egeria

Egeria core
https://egeria-project.org
Apache License 2.0
807 stars 261 forks source link

Kubernetes on Ubuntu 20.04 & Fedora 34 issues #5206

Closed exonianp closed 3 years ago

exonianp commented 3 years ago

In a clean Ubuntu (with no other software installed other than curl) the set up of the kubernetes was performed from scratch following these instructions:,

Tried to follow through the instructions.

The outcome is as shown here:

image

Kafka, zookeeper, platform-0 machines that require persistent volume storage come with this error: 0/1 nodes are available: 1 pod has unbound immediate PersistentVolumeClaims.

config-xz8bh comes with this error: _Failed to create pod sandbox: rpc error: code = Unknown desc = failed to set up sandbox container "acd1c9300303b61d4806b2adc29ab8b74545819b459a8da197ab90eebe3ea167" network for pod "egeria-base-config-xz8bh": networkPlugin cni failed to set up pod "egeria-base-config-xz8bhdefault" network: open /run/flannel/subnet.env: no such file or directory

The machine that presentation-647cbcc9fc-rmm7s machine only worked after issuing this instruction: kubectl taint nodes egeria node-role.kubernetes.io/master-

I think (because I am still investigating) that Ubuntu requires explicit configuration of the Persistent Volumes in the yaml. Else it fails - e.g. where exactly the volumes will be created and that that have been granted sufficient privileges.

In the lab example the situation is as follows: image

So far it also seems that other installed software, such as ansible, can prevent such events.

https://www.linode.com/docs/guides/deploy-minio-on-kubernetes-using-kubespray-and-ansible/

I am happy to create a documentation about the exact installation steps for Ubuntu.

exonianp commented 3 years ago

With the hope that the issue was ubuntu 20.04 specific, I installed Fedora 34 and the situation is identical.

image

I hope that an expert on the matter will enlighten us. Maybe there is a configuration command that allows (but by default prevents) for the creation of volumes in persistent volume claims.

planetf1 commented 3 years ago

This is as expected, though some clarification in the docs to explain it's dependent on persistent storage could be useful.

Many k8s requirements will automatically allocate storage. In particular public cloud - For example openshift does, IBM IKS. It does this using a 'storage provisioner'

In a local environment using microk8s this is enabled by microk8s.enable storage and I think (not verified) docker for windows/mac enables a provisioner by default

So you have a choice of either:

planetf1 commented 3 years ago

Set up cephfs perhaps? https://ubuntu.com/kubernetes/docs/storage

exonianp commented 3 years ago

OMG!

Yes thank you very much Nigel - this ought to be it!