`vmrun getGuestIPAddress` returns incorrect IP when many interfaces are present

gtirloni commented 8 years ago

Host: Windows 10 64-bit Hypervisor: VMware Workstation 12.1 Guest: Fedora 23/24

If other network devices are present on the Guest, vmrun getGuestIPAddress returns incorrect address. For instance, the libvirtd daemon created virbr0 and virbr0-nic interface when it's running and the IP address of virbr0 will be returned (e.g. 192.168.122.x, which is not part of any VMware virtual network).

Uninstalling libvirtd and rebooting, or deleting the virbr0 interface works, vmrun getGuestIPAddress returns the correct IP address (in this case, in the 192.168.190.0/24 network, as expected).

ravindravmw commented 8 years ago

I don't think we make any guarantees about the IP address we return from the guest. Why do you have any assumptions about the IP address? Did you find it documented anywhere?

gtirloni commented 8 years ago

The assumption I am making is that the guest IP returned will be one which is part of the VMware-managed networks and thus useful for something.

This is not documented any where and I'm not saying you are breaking any contracts. I'm just reporting a situation that could be improved.

If the IP address is selected at random, you might as well pick the loopback interface and always return 127.0.0.1.

ravindravmw commented 8 years ago

@gtirloni Thanks for the report. I have opened a VMware internal bug to track the fix for this issue.

ravindravmw commented 8 years ago

@gtirloni Could you please share ifconfig output from your VM for our testing purpose?

gtirloni commented 8 years ago

ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 816  bytes 68397 (66.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 471  bytes 45931 (44.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:fe:cf:be:3d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

> vmrun.exe getGuestIPAddress centos7.vmx -wait
172.17.0.1

If I rename the virbr0 to ens161:

[root@localhost ~]# ip link set virbr0 name ens161
[root@localhost ~]# ip link set ens161 up
[root@localhost ~]# ifconfig -a
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 1351  bytes 111146 (108.5 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 765  bytes 72727 (71.0 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens161: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:fe:cf:be:3d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Result:

> vmrun.exe getGuestIPAddress centos7.vmx -wait
172.17.0.1

If I rename it to docker0:

[root@localhost ~]# ip link set ens161 down
[root@localhost ~]# ip link set ens161 name docker0
[root@localhost ~]# ip link set docker0 up
[root@localhost ~]# ifconfig -a
docker0: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:fe:cf:be:3d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 1591  bytes 130008 (126.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 886  bytes 83673 (81.7 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Result:

> vmrun.exe getGuestIPAddress centos7.vmx -wait
192.168.109.145

Likewise, a completely made up name:

[root@localhost ~]# ip link set docker0 down
[root@localhost ~]# ip link set docker0 name somethingcrazy0
[root@localhost ~]# ip link set somethingcrazy0
[root@localhost ~]# ifconfig -a
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 1809  bytes 147432 (143.9 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1006  bytes 94481 (92.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

somethingcrazy0: flags=4098<BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:fe:cf:be:3d  txqueuelen 0  (Ethernet)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Result:

> vmrun.exe getGuestIPAddress centos7.vmx -wait
172.17.0.1

Setting it to ensX now:

[root@localhost ~]# ip link set somethingcrazy0 down
[root@localhost ~]# ip link set somethingcrazy0 name ens32
[root@localhost ~]# ip link set ens32 up
[root@localhost ~]# ifconfig -a
ens32: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:fe:cf:be:3d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 2090  bytes 169761 (165.7 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1149  bytes 107367 (104.8 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Result:

> vmrun.exe getGuestIPAddress centos7.vmx -wait
172.17.0.1

Setting it to loX:

[root@localhost ~]# ip link set ens32 down
[root@localhost ~]# ip link set ens32 name lo1
[root@localhost ~]# ip link set lo1 up
[root@localhost ~]# ifconfig -a
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 2277  bytes 184511 (180.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1247  bytes 116147 (113.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo1: flags=4099<UP,BROADCAST,MULTICAST>  mtu 1500
        inet 172.17.0.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 02:42:fe:cf:be:3d  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Result:

> vmrun.exe getGuestIPAddress centos7.vmx -wait
172.17.0.1

Deleting all interface except the NAT'ed and lo:

[root@localhost ~]# ip link delete lo1
[root@localhost ~]# ifconfig -a
ens160: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.109.145  netmask 255.255.255.0  broadcast 192.168.109.255
        inet6 fe80::20c:29ff:fe3a:558e  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:3a:55:8e  txqueuelen 1000  (Ethernet)
        RX packets 2414  bytes 195258 (190.6 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1314  bytes 122155 (119.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10<host>
        loop  txqueuelen 0  (Local Loopback)
        RX packets 4  bytes 340 (340.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 4  bytes 340 (340.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Result:

> vmrun.exe getGuestIPAddress centos7.vmx -wait
192.168.109.145

sbourlon commented 8 years ago

Same problem with a VM running a docker daemon.

ravindravmw commented 8 years ago

Thanks @gtirloni and @sbourlon. I have opened a VMware internal bug to track this issue.

sbourlon commented 8 years ago

Thank you @ravindravmw for the update.

It worked for me with this settings on Fusion in Host-only:

$ ip -4 a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eno16777984: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    inet 192.168.156.128/24 brd 192.168.156.255 scope global dynamic eno16777984
       valid_lft 1748sec preferred_lft 1748sec
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever

% vmrun getGuestIPAddress "~/Documents/Virtual Machines.localized/87.vmwarevm/87.vmx"
192.168.156.128

Version of open-vm-tools:

$ rpm -qa|grep open-vm-tools
open-vm-tools-9.10.2-5.el7_2.x86_64

My problem was actually not related to open-vm-tools but the network interface name changed during the first boot of my OVA and no network manager rules matched the new name.

sbourlon commented 8 years ago

Interesting. After my OVA setups all the network interfaces, it does not work anymore:

[stefan@localhost ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eno16777984: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
    link/ether 00:0c:29:f5:e5:22 brd ff:ff:ff:ff:ff:ff
    inet 192.168.156.128/24 brd 192.168.156.255 scope global dynamic eno16777984
       valid_lft 1427sec preferred_lft 1427sec
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
    link/ether 02:42:ed:8e:d8:af brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 scope global docker0
       valid_lft forever preferred_lft forever
17: br-a01c4bc83b0c: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP
    link/ether 02:42:35:f3:87:55 brd ff:ff:ff:ff:ff:ff
    inet 172.18.0.1/16 scope global br-a01c4bc83b0c
       valid_lft forever preferred_lft forever
19: vethe1663cf@if18: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-a01c4bc83b0c state UP
    link/ether ee:08:1d:ca:26:6b brd ff:ff:ff:ff:ff:ff link-netnsid 0
21: veth0204d50@if20: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-a01c4bc83b0c state UP
    link/ether 9e:fc:4a:c8:5a:fd brd ff:ff:ff:ff:ff:ff link-netnsid 1
23: veth1d82ff3@if22: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-a01c4bc83b0c state UP
    link/ether e2:9f:cd:a2:28:e5 brd ff:ff:ff:ff:ff:ff link-netnsid 2
25: vethce2587b@if24: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-a01c4bc83b0c state UP
    link/ether 7a:11:90:92:d7:8a brd ff:ff:ff:ff:ff:ff link-netnsid 3
27: vethbc0b9f2@if26: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-a01c4bc83b0c state UP
    link/ether c6:56:66:da:3e:38 brd ff:ff:ff:ff:ff:ff link-netnsid 4
29: veth4176a85@if28: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-a01c4bc83b0c state UP
    link/ether 92:4c:fb:4c:77:35 brd ff:ff:ff:ff:ff:ff link-netnsid 5

% vmrun getGuestIPAddress "~/Documents/Virtual Machines.localized/87.vmwarevm/87.vmx"
172.18.0.1

sbourlon commented 8 years ago

@ravindravmw Any update on the VMware internal bug you opened?

ravindravmw commented 8 years ago

@sbourlon The bug is being targeted for a release sometime next year. BTW, I'd like to understand if you are referring to a different issue when you said this in your 3rd last comment:

My problem was actually not related to open-vm-tools but the network interface name changed during the first boot of my OVA and no network manager rules matched the new name.

I'm not sure what were you referring to in this comment.

sbourlon commented 8 years ago

@ravindravmw I thought is was actually working but after the provisioning of my VM added some network interfaces, the IP returned by vmrun getGuestIPAddress became incorrect.

julien-duponchelle commented 8 years ago

How we workaround this at GNS3. This is not solution for everyone but could help some people reading this thread.

For a Debian based VM.

In the VM we create a file /etc/network/if-up.d/gns3-ifup with chmod 755 and owned by root:

#!/bin/sh

command -v vmtoolsd >/dev/null 2>&1 || exit 0

IP=`ifconfig $IFACE | grep 'inet addr:' | cut -d: -f2 | cut -d' ' -f1`

vmtoolsd --cmd "info-set guestinfo.gns3.$IFACE $IP"

This script will be execute each time our VM got an IP and it will set a VMware variable gns3.NAME_OF_INTERFACE with the IP attributed to the VM

From the outside you can now get this IP with vmrun:

vmrun -T ws readVariable "GNS3 VM.vmx" guestVar gns3.eth0

sbourlon commented 8 years ago

thank you @noplay!

tknerr commented 7 years ago

Similar to @noplay's approach I tried setting vmtoolsd --cmd "info-set guestinfo.ip $IP" (where $IP is set to the ip address of the correct adapter, e.g. eth0)

This seems to work for a moment, and all of

vmtoolsd --cmd "info-get guestinfo.ip"
and vmrun readVariable <vmx> guestVar ip
as well as vmrun getGuestIPAddress <vmx> (!)

...will return $IP when asked for it.

However, this works only for a couple of seconds, and within less than a minute it returns the old ip address (of the wrong adapter) again.

Any ideas how to make this setting permanent, or which other process resets it to the wrong adapter's ip address again?

lemke1458 commented 7 years ago

vmtoolsd sets "guestinfo.ip" via its guestInfo plugin every 30 seconds. Moving that plugin out of the way would make your override stick, but you'd also lose everything else the guestInfo plugin reports (stats, full NIC/IP stack, disk info, etc).

daBONDi commented 7 years ago

Got still the same problem with the vmware workstation provider :-( The first boot is ok, because my vagrant box image got only 1 adapter so it cannot wrong connect, but second provision run failed because vmrun "getGuestIPAddress" returns wrong ip

asoki commented 6 years ago

got the same problem today with vagrant 2.0.2.

my interfaces file on the guest:

cat /etc/network/interfaces 
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
allow-hotplug eth0
iface eth0 inet dhcp

#VAGRANT-BEGIN
# The contents below are automatically generated by Vagrant. Do not modify.
auto eth1
iface eth1 inet static
      address 192.168.116.80
      netmask 255.255.255.0
#VAGRANT-END

the IP 192.168.116.80 coming from the Vagrantfile as "foo_config.vm.network "private_network", ip: "192.168.116.80"

editing /etc/network/interfaces and deleting the VAGRANT-* block, and making a vagrant reload solved the problem for now.

my two cents: when i inspect the the /etc/network/interfaces file, i see that the vagrant interface is marked auto eth1

I think, one cause of this "which interface is the right interface" problem is that:

stopping the guest with vagrant halt leaves the interfaces file as is, with the active VAGRANT block
the vagrant generated interface is marked with "auto"

so on the next boot all defined interfaces become automatically active and causing the problem of finding the "right" interface.

maybe a change in the inner works of vagrant (not marking the generated interface as "auto") would solve the problem, or would make finding the interface easier.

johnwvmw commented 6 years ago

@asoki

You have not indicated the version(s) of open-vm-tools that you are using.

If you are using tools 10.2.0 or later, you can specify the primary NIC device in the tools.conf configuration file. See the "primary-nics" option on the following web-page in the VMware Tools User Guide

https://docs.vmware.com/en/VMware-Tools/10.2.0/com.vmware.vsphere.vmwaretools.doc/GUID-ECCF9D01-3666-40CE-B9FD-7EE0738AB5D9.html

stanguturi commented 5 years ago

getGuestIPAddress option in vmrun looks for 'guestinfo.ip' variable for the VM and reports it to the user. The 'guestinfo.ip' variable is populated by the 'guestinfo' plugin in tools.

Check GuestInfoGetPrimaryIP function in https://github.com/vmware/open-vm-tools/blob/master/open-vm-tools/lib/nicInfo/nicInfoPosix.c for the logic used to figure out what is the primary IP in the VM.

https://docs.vmware.com/en/VMware-Tools/10.2.0/com.vmware.vsphere.vmwaretools.doc/GUID-ECCF9D01-3666-40CE-B9FD-7EE0738AB5D9.html

Also, as mentioned in the previous comment, starting from tools 10.2.0 the user can always override the primary nic using an option in the tools.conf Please check it out.

Thanks

vmware / open-vm-tools

`vmrun getGuestIPAddress` returns incorrect IP when many interfaces are present #93