External interface

[cgoguyer]

Hello, it seems that the external interface is not working anymore as described: https://www.anstack.com/blog/2020/08/25/KubeInit-External-access-for-OpenShift-OKD-deployments-with-Libvirt.html So, how to setup an external interface to access okd-services/console outside internal libvirt network ?

[gmarcy]

any chance you still have that cluster around you wanted to add external access to?

on the bastion host there should be a script in the root home folder called create-external-ingress.sh

running that should give you external access through that host. if it doesn't let us know, it's something I added in place of the old external access mechanism that I could never get working consistently and I still owe Carlos documentation help for that change.

[ccamacho]

@cgoguyer we are working to add some new features among them the no need of having this additional bridged interface, so far you could deploy 1.0.2.kubeinit-2021-03-19-18-05-19 and it should work as it was before.

[cgoguyer]

@gmarcy There is no "create-external-ingress.sh" script on service host.

@ccamacho I'using the latest commit (>1.0.2) but as there is no bridge (nor external naming zone) on service host, How do you manage to get access to "*.apps.okdcluster.kubeinit.local" from laptops outside 10.0.0.0 private network ? I've seen some variable about that but seems not been used (kubeinit_common_external_access_enabled, kubeinit_inventory_network_bridge_external, ...)

[gmarcy]

@cgoguyer the service host and the first hypervisor (bastion) are different places, the script should be on the hv where the cluster network is created, default name nyctea, not the service host at 10.0.0.100

[cgoguyer]

@ccamacho Ok, I got it. With version 1.0.2.kubeinit-2021-03-19-18-05-19, all works as before. Changes are coming after this tag.

But after the reboot of service node, OKD cluster is not accessible anymore:

[root@okd-service-01 ~]# oc get nodes
Unable to connect to the server: dial tcp: lookup api.okd4.kubeinit.local: no such host

Any idea ?

[cgoguyer]

@gmarcy Ok, finally i found the script on the hypervisor (is it linked to 'kubeinit_common_external_access_enabled': 'true' ?), but i get an error as I have multiple interfaces (including libvirt/podman) on the hypervisor.

sed: can't read 10.x.x.x: No such file or directory
sed: can't read 10.x.x.x: No such file or directory
sed: can't read 10.x.x.x; };: No such file or directory

$(hostname --ip-address) return multiple values.

[gmarcy]

ah, Carlos had a fix for that which I need to pick up. something about finding the one used for the default route. can you confirm that if you change the script to use the interface ip that would be used when you connect to somewhere like 8.8.8.8 that the script sets up the external access properly?

[cgoguyer]

@ccamacho Ok, I got it. With version 1.0.2.kubeinit-2021-03-19-18-05-19, all works as before. Changes are coming after this tag.

But after the reboot of service node, OKD cluster is not accessible anymore:

[root@okd-service-01 ~]# oc get nodes
Unable to connect to the server: dial tcp: lookup api.okd4.kubeinit.local: no such host

Any idea ?

@ccamacho The issue is coming from "ifcfg-eth0", DNS1=10.0.0.254 was commented and replaced by DNS1=8.8.8.8. So after reboot, I have a wrong nameserver in resolv.conf and cluster is not accessible. And default route should be yes in "ifcfg-eth1" and no in "ifcfg-eth0", in version 1.0.2 it is reversed.

I saw that you have dropped the bridge template in the latest version but the issue with DNS may still cause cluster not accessible form service host.

[cgoguyer]

ah, Carlos had a fix for that which I need to pick up. something about finding the one used for the default route. can you confirm that if you change the script to use the interface ip that would be used when you connect to somewhere like 8.8.8.8 that the script sets up the external access properly?

@gmarcy After hardcoded the "KUBEINIT_INGRESS_IP" variable, the script worked fine and the pod is created. But I have not seen any ports published for the kubeinit-ingress-bind pod, so how do you configure dns access from external desktops ?

[gmarcy]

from the external desktops I usually add nameserver <KUBEINIT_INGRESS_IP>. on my current external desktop I have

search okdcluster.kubeinit.local
nameserver 192.168.0.2
nameserver 8.8.8.8

where 192.168.0.2 is the address of nyctea, the hypervisor where I ran the create-external-ingress.sh script

[gmarcy]

@cgoguyer can you confirm that this works better than hostname --ip-address in your environment?

KUBEINIT_INGRESS_IP=$(ip route get "8.8.8.8" | grep -Po '(?<=(src )).*(?= uid)')

[cgoguyer]

@cgoguyer can you confirm that this works better than hostname --ip-address in your environment?

KUBEINIT_INGRESS_IP=$(ip route get "8.8.8.8" | grep -Po '(?<=(src )).*(?= uid)')

It is better, but in my case I get the management address, but not the one used for application network. It would be great to add a variable to override this setup.