search

darxkies / k8s-tew

Kubernetes - The Easier Way
GNU General Public License v3.0
307 stars 38 forks source link

Support for MetalLB #9

Closed leonj1 closed 5 years ago

leonj1 commented 5 years ago

What's the process to install additional dependencies to the cluster upon creation? Example, I'm interested in MetalLB so Services can have External IPs when they get created. At the moment, upon cluster creation (ubuntu-multi) everything is properly created, but the Services do not have External IPs. If I install MetalLB after cluster creation, existing Services do not get External IPs (that may be a limitation of MetalLB, therefore I'm interested in installing that early.

darxkies commented 5 years ago

Sorry for the late reply.

Currently, there is no CLI support for managing dependencies. I will add that with a future release. Meanwhile, you can do that manually by editing the content of the file assets/etc/k8s-tew/config.yaml. Look for the section "commands", which contains the shell commands that are executed to set up a cluster. MetalLB can be installed using helm. So you could add a "command" to run helm and install MetalLB. The label has to be set to "bootstrapper" and you should omit the "features" entry.

Later on, I will add native support for MetalLB.

An alternative to using MetalLB is to use k8s-tew's Virtual/Floating IPs for controllers & workers. If specified, the one worker IP can be shared among all workers. And the traffic is forwarded to this Virtual/Floating IP.

darxkies commented 5 years ago

Trying to wrap my head around MetalLB. Do you have any use cases or some configurations that you want implemented?

leonj1 commented 5 years ago

Sure thing. The goal is to get an External IP to a Service

$ k get svc -n gru
NAME            TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
gru-api         LoadBalancer   10.96.10.235     10.1.1.220    8686:30994/TCP   8d
gru-videos-ui   LoadBalancer   10.105.182.173   10.1.1.221    9080:31764/TCP   1d

From my experiments, MetalLB needs to be installed ahead of time, before any Service gets created. If I create a Service and then install MetalLB an external IP was not assigned. The goal is to make Services accessible to the outside world (outside the K8s cluster).

My steps:

# install metallb
kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.7.3/manifests/metallb.yaml

# configure MetalLB via ConfigMap to provide it a subnet of IPs to allocate for services
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 192.168.1.240/28

# source: https://medium.com/@JockDaRock/kubernetes-metal-lb-for-on-prem-baremetal-cluster-in-10-minutes-c2eaeb3fe813
darxkies commented 5 years ago

Great. That's a starting point. I will take a look at that in the next couple of days. Hopefully there will be no issues with Calico (https://metallb.universe.tf/configuration/calico/).

darxkies commented 5 years ago

There is a new release with MetalLB support:

https://github.com/darxkies/k8s-tew/releases/tag/2.2.4

darxkies commented 5 years ago

Have you had the chance to take a look at it? Is it what you needed?

leonj1 commented 5 years ago

Spinning up the cluster completed successfully, and I did see the installation of MetalLB. But I'm having some difficulty using kubectl to manage the cluster.

cd setup/ubuntu-multi-node
make run
$ ./k8s-tew environment
ERRO[0000] Failed initializing                           error="Unsupported config version '2.1.0'"
# I run this before using eval to see what its going to do, but got an error this time

Not sure how to proceed, admittedly unlikely related, error.

darxkies commented 5 years ago

It looks like you used two different k8s-tew versions.

If you run which k8s-tew; k8s-tew what is the output?

leonj1 commented 5 years ago

That was the problem. I made sure to use the latest version and MetalLB worked great!

$ k get svc
NAME         TYPE           CLUSTER-IP    EXTERNAL-IP       PORT(S)          AGE
kubernetes   ClusterIP      10.32.0.1     <none>            443/TCP          29m
nginx        LoadBalancer   10.32.0.166   192.168.120.201   8080:30048/TCP   3s
$ curl "http://192.168.120.201:8080"
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
...
<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Thanks for the quick turnaround!