search

karmab / kcli

Management tool for libvirt/aws/gcp/kubevirt/openstack/ovirt/vsphere/packet
https://kcli.readthedocs.io/en/latest/
Apache License 2.0
474 stars 133 forks source link

`kcli list vm` not correctly get the IP #645

Closed nsu700 closed 4 months ago

nsu700 commented 4 months ago

Found the IP is not the correct one for my vSphere VM 924d8-worker-3

[lab-user@bastion-924d8 ~]$ kcli list vm
+------------------+--------+-----------------+------------------------------------+------+---------+
|       Name       | Status |        Ip       |               Source               | Plan | Profile |
+------------------+--------+-----------------+------------------------------------+------+---------+
| 924d8-ctlplane-0 |   up   | 192.168.172.140 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
| 924d8-ctlplane-1 |   up   |  192.168.172.42 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
| 924d8-ctlplane-2 |   up   |  192.168.172.75 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-0  |   up   | 192.168.172.141 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-1  |   up   | 192.168.172.142 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-2  |   up   |  192.168.172.76 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-3  |   up   |    10.134.2.2   | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  bastion-924d8   |   up   |  192.168.172.10 |                                    |      |         |
+------------------+--------+-----------------+------------------------------------+------+---------+

The worker-3 should be 192.168.172.143, but in the above output, it is 10.134.2.2, seems it just grab the ovn-k8s-mp0 address, but not the br-ex

[core@924d8-worker-3 ~]$ ip a s
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: ens192: <BROADCAST,MULTICAST,ALLMULTI,UP,LOWER_UP> mtu 1500 qdisc mq master ovs-system state UP group default qlen 1000
    link/ether 00:50:56:a2:cb:74 brd ff:ff:ff:ff:ff:ff
3: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether d6:19:3c:18:d3:53 brd ff:ff:ff:ff:ff:ff
4: genev_sys_6081: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 65000 qdisc noqueue master ovs-system state UNKNOWN group default qlen 1000
    link/ether 96:a6:65:21:a1:18 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::94a6:65ff:fe21:a118/64 scope link
       valid_lft forever preferred_lft forever
5: br-int: <BROADCAST,MULTICAST> mtu 1400 qdisc noop state DOWN group default qlen 1000
    link/ether 6e:8b:55:0d:51:db brd ff:ff:ff:ff:ff:ff
6: ovn-k8s-mp0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN group default qlen 1000
    link/ether 92:db:67:b7:56:81 brd ff:ff:ff:ff:ff:ff
    inet 10.134.2.2/23 brd 10.134.3.255 scope global ovn-k8s-mp0
       valid_lft forever preferred_lft forever
    inet6 fe80::90db:67ff:feb7:5681/64 scope link
       valid_lft forever preferred_lft forever

My environment is:

Attach my paramfile for reference, the worker-2 and worker-3 were added after OCP provisoned with command kcli scale kube openshift --paramfile parameters.yml -w 4 openshift :

[lab-user@bastion-924d8 ~]$ cat parameters.yml
cluster: 924d8
domain: dynamic.test.home.com
version: stable
tag: '4.12.26'
ctlplanes: 3
workers: 2
memory: 16384
numcpus: 16
network: segment-sandbox-924d8
pool: WorkloadDatastore
api_ip: 192.168.172.201
ingress_ip: 192.168.172.202
service_network_ipv4: 172.30.0.0/16
pull_secret: ./pull-secret.txt
karmab commented 4 months ago

I'm not sure we can influence which ip gets reported primarily although you should see the additional ones in kcli info vm (under ips field). Can you share output of kcli --debug info vm 924d8-worker-3

nsu700 commented 4 months ago

Yes, please check the attachment Found it not always get the wrong IP, I have restarted worker-0, worker-2 and worker-3, now only the worker-0 returns the correct IP

[lab-user@bastion-924d8 ~]$ kcli list vm
+------------------+--------+-----------------+------------------------------------+------+---------+
|       Name       | Status |        Ip       |               Source               | Plan | Profile |
+------------------+--------+-----------------+------------------------------------+------+---------+
| 924d8-ctlplane-0 |   up   | 192.168.172.140 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
| 924d8-ctlplane-1 |   up   |  192.168.172.42 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
| 924d8-ctlplane-2 |   up   |  192.168.172.75 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-0  |   up   | 192.168.172.141 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-1  |   up   | 192.168.172.142 | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-2  |   up   |    10.133.2.2   | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  924d8-worker-3  |   up   |    10.134.2.2   | rhcos-412.86.202306132230-0-vmware | ocp  |  kvirt  |
|  bastion-924d8   |   up   |  192.168.172.10 |                                    |      |         |
+------------------+--------+-----------------+------------------------------------+------+---------+
[lab-user@bastion-924d8 ~]$

debug-vm.txt

karmab commented 4 months ago

sadly, the debug file doesnt contain the kind of information I was looking for. As I am currently missing a vsphere platform where to troubleshoot this properly, please reach out on slack and we can do a tmate session on this

karmab commented 4 months ago

https://github.com/karmab/kcli/commit/eac69aec70722f07c9ca4a479ee26ae534676f04 should address it, provided you use default sdn range in your openshift cluster. I can't provide a cleaner fix because vsphere api doesnt provide a better way to identify which ips are valid