vmware / open-vm-tools

Official repository of VMware open-vm-tools project
http://sourceforge.net/projects/open-vm-tools/
2.28k stars 428 forks source link

Centos 7.4 x64 not correctly recognised on ESXi 5.5U3 #197

Open LapinFou opened 7 years ago

LapinFou commented 7 years ago

Hi folks,

When I updated the CentOS 7.3 VM at my work, I got a major issue. vSphere doesn't correctly recognized CentOS 7.4. The consequence is Horizon 7 (v7.3.1 build-6760913) doesn't recognized either the VMs. You can see on the attached snapshot before and after the CentOS update. tmp

The only workaround I found is (not supported nor recommended by VMware): yum –y install gcc-c++ kernel-devel kernel-headers yum –y remove open-vm-tools install latest VMware Tools → VMwareTools-10.1.15-6677369.tar.gz install latest VMware Horizon Agent → VMware-horizonagent-linux-x86_64-7.3.0-6604962.tar.gz At this point, it still didn't fix the issue. Then, I launch a VMware Tools update from the vSphere interface. This will downgrade the VMware Tool to the version 9.4.15.48277 (build-2827462) I tried to directly installed the VMware Tool provided by vSphere, but it failed to compile some module (probably to old version). Installing the VMwareTools v10.1.15 (up-to-date) did compile fine. So during the "downgrade" process, the VMwareTools v9.4.15 didn't recompile anything (this is my guess).

So, it is working, but my VM hang 50s during the boot sequence (waiting some VMware stuff). Despite this problem, this workaround made Horizon 7 happy.

LapinFou commented 7 years ago

I forgot to mention that the package redhat-lsb is installed.

jonathanvmw commented 7 years ago

Thanks for reporting this issue.

What version of vSphere are you running?

LapinFou commented 7 years ago

vSphere Client v5.5.0 Build 3024345 VMware vCenter Server v5.5.0 Build 4180647

LapinFou commented 7 years ago

BTW, I tried to do a fresh install from the latest CentOS7 ISO, but I got exactly the same issue.

jonathanvmw commented 7 years ago

My understanding is that the screenshot is showing how it looks after the Centos Update but before you applied the workaround - is that correct? What is being reported for the OS type for Centos 7.4 VMs after you applied the workaround - is it the same as shown in the snapshot, or something different?

Thanks.

LapinFou commented 7 years ago

Hi,

You understanding is correct.

Using the included open-vm-tools on CentOS 7.3 was working fine. vSphere reported the Guest OS as CentoOS 4/5/6/7 (64bits). Same thing when applying the workaround on CentOS 7.4, vSphere will display the Guest OS as CentoOS 4/5/6/7 (64bits).

If I do upgrade from CentOS 7.3 to 7.4 or do a fresh install of CentOS 7.4, vSphere will report the Guest OS as Linux 3.10.0-693.2.2.el7.x86_64 CentOS Linux release 7.4.1708 (Core).

LapinFou commented 7 years ago

Here is a new snapshot. Top pic is with the workaround. Bottom pic is what I got after upgrading to CentOS 7.4 or doing a fresh 7.4 install. tmp

On the bottom pic, you can see that vSphere detected the presence of VMware Tools → Exécution (Tiers /indépendant). However vSphere doesn't correctly report the Guest OS.

SE client field in French must be translated as Guest OS in English.

jonathanvmw commented 7 years ago

Thanks for the additional information.

There is a known problem that vSphere 6.0 and earlier does not properly identify some newer guest OSes like CentOS 7.4. Upgrading to newer releases of vSphere and VMware Tools will resolve the problem. We are not planning to backport awareness of the new guest OSes to the older vSphere releases.

I've filed a bug internally for your report since it's new information that Horizon View does not work in this scenario and that reverting to an earlier version of open-vm-tools allows the OS to be identified correctly.

LapinFou commented 7 years ago

This is a big issue for me. Our VMware provider said they don't want upgrade vSphere to the version 6.5 for stability reasons... They are sticking to vSphere v5.5U3. Since it was working fine with CentOS 7.3, there is no chance to "patch" open-vm-tools? :cold_sweat:

jonathanvmw commented 7 years ago

Offhand I don't know. The normal way this works relies on changes to vSphere.

I've added your last comment to the internal bug.

jonathanvmw commented 7 years ago

I have two suggestions.

FIrst, file a support request with VMware support. This will make it more likely that your problem gets attention within VMware, though it’s no guarantee that it will be fixed.

Second, as an alternate workaround, you could try changing the GuestOS type of the VM to “centos.” It’s not supported, we think it might work, thereby allowing you to update tools and avoid the 50s hang you’re seeing. It’s possible that a GuestOS value of “rhel7-64” would also work. However, please note that nobody has actually tried either of these. You can modify the GuestOS field either through the vi client or by editing the “.vmx” file for the VM. If you try this as a workaround, make sure you record the previous value so you can restore it if needed.

LapinFou commented 7 years ago

@jonathanvmw Thanks you so much for trying to help. I fill up a ticket to VMware support. I also will try your workaround. I will post the results next week (I'm right now out of my office for business). I will keep you inform.

LapinFou commented 7 years ago

@jonathanvmw I tried the trick to modify the .vmx file. Unfortunately, I got an erro message which said "rhel7_64Guest" is not supported.

LapinFou commented 7 years ago

EDIT: ESXi 5.5U3 support only up to RHEL6. This is probably why I got this error messafe

jonathanvmw commented 7 years ago

Did you try changing GuestOS to "centos"?

Also, please let me know the support request number once you've filed it. Thanks.

LapinFou commented 7 years ago

Since I'm using EFI Bios, the VM will not boot with centos parameter. tmp

Obiwanbde commented 7 years ago

Hi all,

It is very nice to find this thread. I have tried to search for the error message but not succeeded until just checking "CentOS Linux release 7.4.1708 vsphere support". I could not find the bug filled by LapinFou, could you please share the link ?

OK, I have the very same issue for ESXi 6.0.0 Update 2 (Build 4192238).

I have recently downloaded Centos 7 - 1708 and tried to install in a VM. Setting Guest OS Version as CentOS 4/5/6/7 (64-bit) or Red Hat Enterprise Linux 7 (64-bit) or Other Linux (64-bit) did not help.

Kernel of 3.10.0-693.el7.x86_64 or updated Kernel of 3.10.0-693.5.2.el7.x86_64 not changed anything.

Guest OS is shown as CentOS 4/5/6/7 (64-bit) but,

It always says that : "The configured guest OS (CentOS 4/5/6/7 (64-bit)) for this virtual machine does not match the guest that is currently running (Linux 3.10.0-693.el7.x86_64 CentOS Linux release 7.4.1708 (Core) ). You should specify the correct guest OS to allow for guest-specific optimizations."

VMware tools reported as installed and running and shown version is 2147483647.

And installed open-vm-tools is open-vm-tools.x86_64 - 10.1.5-3.el7-@anaconda

I have tried to change guestOS = "centos-64" in .vmx to "centos" did not help and resulted as "CentOS 4/5/6 (32-bit)"

centos vsphere problem

As like, LapinFou, my provider does not want to perform update for ESXi 6.0, even I do not sure if the update helps because Update 3 is released earlier than this Centos 7.4 - 1708. And yes similarly, I have been blocked for installing a new CentOS VM and upgrading many CentOS 7 servers.

Any help appreciated,

Thanks

jonathanvmw commented 7 years ago

@Obiwanbde

Thanks for reporting the problem. As mentioned above, there is an internal bug report for this issue, and I will add your comment to it. I suggest that you also file a support request with VMware and let me know what the SR # is once you do that.

You asked for an external link for the bug report; to my knowledge there isn't anything externally visible other than this.

Obiwanbde commented 7 years ago

@jonathanvmw

Thanks for prompt response and adding comment to internal request.

Will try to get access to support pages for filing a request. I have some issues with support agreement.

In parallel, I have re-evaluated to upgrade to 6.0 U3 or even 6.5, although it would be quite a job to do that and even I do not sure about if it will solve the issue

What I see is until Kernel 514 we do not have any problems but after that(693) this issue happens on the same vsphere. And it seems that 693 contains a security fix.

What reports or what interprets incorrectly ? I have also seen that some people tried to install that level of Kernel on VMware Workstation 12.5 and got problems.

Should I try the "not recommended", uninstall open-vm-tools and install VMware Tools ? new one or the one comes from my vsphere ?

On Thu, Nov 9, 2017 at 1:10 AM, jonathanvmw notifications@github.com wrote:

@Obiwanbde https://github.com/obiwanbde

Thanks for reporting the problem. As mentioned above, there is an internal bug report for this issue, and I will add your comment to it. I suggest that you also file a support request with VMware and let me know what the SR # is once you do that.

You asked for an external link for the bug report; to my knowledge there isn't anything externally visible other than this.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vmware/open-vm-tools/issues/197#issuecomment-342977719, or mute the thread https://github.com/notifications/unsubscribe-auth/AQ-lfQHVN5IjzS9uSpnWQKSokvVWPzfZks5s0ibOgaJpZM4PwvrL .

Obiwanbde commented 7 years ago

@jonathanvmw -update- I have downloaded 10.1.15(last release), built it(some extra efforts needed for missing packages on a test system) and reboot.

Now vmtoolsd -v shows as VMware Tools daemon, version 10.1.15.65452 (build-6677369)

But the warning on vsphere is still as the same

LapinFou commented 7 years ago

On my side, it seems our IT provider didn't fill-up a ticket... 😞 I will keep you in touch.

Obiwanbde commented 7 years ago

@jonathanvmw -update-

I cannot reach VMware support due to provider, maybe they do not have proper support, so I cannot file a support request.

But I do something different, which makes me not happy.

I have downloaded ESXi 6.5U1 evaluation for 60days and install it on my VMWare Workstation as a VM and run it and install Centos7.4 1708 Minimal on a VM in that ESXi. So, Centos7.4 in VM in ESXi6.5u1(VM) in VMware Workstation in Windows.

After initial install, I see Centos minimal install does not have open-vm-tools,(I see the same 10.1.5), I have installed it with yum and I see no warnings on vsphere management panel.

image

So, ESXi 6.5 built on last September has no problems with Centos7.4 with updated kernel.

But we have problems for having an upgraded ESXi for our real environments. Upgraded kernel of 3.10.0-693.el7.x86_64 creates problems with 5.5U3/6.0U1.

I know you have said, it needs to be fixed on VMware side and you have already added comments to internal bug report.

I would like to ask once again, as we both cannot reach VMware for support request and upgrade of 5.5U3 and 6.0U1 to 6.5U1 seems not easy/quick due to providers and considering others with running older ESXis and having updated their Centos7, could we have a patch/solution in open-vm-tools ?

Thanks again.

jonathanvmw commented 7 years ago

@obiwanbde

Sorry, but as noted in my October 12 comments, this is a known problem with vSphere 6.0 and earlier, and there’s not a way to fix it by patching open-vm-tools.

Obiwanbde commented 7 years ago

@jonathanvmw Thanks for informing.

That means, all Centos7.4 installations on vSphere 6.0 or earlier needed to upgrade their hypervisor to vSphere to 6.5.

And also all Centos7.3 installations should not use/update their kernels to the next level ( 693 ), if they are working in vSphere 6.0 or earlier, or they need to upgrade their hypervisor to vSphere to 6.5.

Am I correct ?

On the other hand, may I ask, how serious is this warning ? how much of guest optimizations are missing when we run with it ? is it a considerable situation for production service ? can we have an insider for this ? what would be the recommendation ?

Again I would not expect quick/easy vSphere upgrades but new kernel already integrated to CentOS distribution since October 2017.

Thanks.

jonathanvmw commented 7 years ago

@obiwanbde

That means, all Centos7.4 installations on vSphere 6.0 or earlier needed to upgrade their hypervisor to vSphere to 6.5.

And also all Centos7.3 installations should not use/update their kernels to the next level ( 693 ), if they are working in vSphere 6.0 or earlier, or they need to upgrade their hypervisor to vSphere to 6.5.

Am I correct ?

Yes.

On the other hand, may I ask, how serious is this warning ? how much of guest optimizations are missing when we run with it ? is it a considerable situation for production service ? can we have an insider for this ? what would be the recommendation ?

You should probably discuss with your provider. I don't know how serious it is. From what I understand, Centos 7.4 on ESXi 6.0 U2 will probably work, but I don't think I would run that environment in production given that it's not supported.

marco2bb2 commented 7 years ago

@jonathanvmw Actually, if I look at official VmWare compatibility list, regardin Centos 7.X (which includes 7.4), I see it is supported for all these versions 5.5 to 6.5U1...

https://www.vmware.com/resources/compatibility/search.php?deviceCategory=software&details=1&operatingSystems=202&page=1&display_interval=10&sortColumn=Partner&sortOrder=Asc&testConfig=16

LapinFou commented 7 years ago

I finally convinced our VMWare provider to migrate our hypervisor to 6.5U1 (planed end of December). I hope CentOS 7.4 will work fine, cos as mentioned by @marco2bb2 CentOS7.4 is supported starting the version 5.5.

Obiwanbde commented 7 years ago

You are lucky then.

As you both stated VMware is supporting 7.4 what is the reason for having incorrect recognition ? Does the VMware have other source than open-vm-tools to gather operating system info from CentOs?

Logically, under the same level of hypervisor if Centos 7.4 with Kernel 514 does not create any problem but when it is upgraded to Kernel 693, hypervisor tells operating system is not recognised and when you upgrade the hypervisor then it recognizes.

Something happens that we do not know in detail for guest-host relation when in 6.0 and earlier.

Altough 6.5 is a year old with an update, it is a job to upgrade it.

marco2bb2 commented 7 years ago

Our problem is that our physical host (ML350 Gen9), with our CPU is supported only with ESXi <= 6.0U3 , so we cannot upgrade further!!

Il 21/11/2017 21:02, Sebastien Charpentier ha scritto:

I finally convinced our VMWare provider to migrate our hypervisor to 6.5U1 (planed end of December). I hope CentOS 7.4 will work fine, cos as mentioned by @marco2bb2 https://github.com/marco2bb2 CentOS7.4 is supported starting the version 5.5.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/vmware/open-vm-tools/issues/197#issuecomment-346144421, or mute the thread https://github.com/notifications/unsubscribe-auth/ADfTtbHhHu-aFrDmwbBwDQvUKG4Rn2sAks5s4yxogaJpZM4PwvrL.

Obiwanbde commented 7 years ago

That is the main question : What would, all users of ESXi 6.0U3 and earlier which cannot upgrade to 6.5; do to use newer, security patched CentOS 7.4 with Kernel 693 and above ?

As far as I know there is no solution yet.

LapinFou commented 7 years ago

With vSphere Client v5.5.0 Build 3024345, there is a workaround. I described it at the begin of this ticket. In my case, our hypervisor (put in production in July of this year) must be reinstalled from scratch, because we are migrating from SAS disks to SSD disks (SSD from HP have been sold out for months now). Anyway, as everyone, I don't understand why VMWare/CentOS doesn't provide a patch. I'm very surprise to not read more complaints about this major issue/regression. Maybe, it does impact only people using Horizon Client?

Obiwanbde commented 7 years ago

@LapinFou What is your Kernel level in CentOS ? I do not have Horizon, it is simply ESXi, 6.00U2 and no solution.

LapinFou commented 7 years ago

@Obiwanbde The current kernel version of my Centos VMs is 3.10.0-693.5.2.el7.x86_64

Obiwanbde commented 7 years ago

Thanks , I have the same and this 693 causes the issue on vSphere 6.0 and earlier. No update in VMware, Centos or even with open-vm-toools. The only known solution is to upgrade 6.0 to 6.5 which I cannot still have it.

Obiwanbde commented 6 years ago

Today I have updated my test system from 3.10.0-693.5.2.el7.x86_64 to 3.10.0-693.11.1.el7.x86_64 and the problem is still the same on vSphere:

"The configured guest OS (CentOS 4/5/6/7 (64-bit)) for this virtual machine does not match the guest that is currently running (Linux 3.10.0-693.11.1.el7.x86_64 CentOS Linux release 7.4.1708 (Core) ). You should specify the correct guest OS to allow for guest-specific optimizations."

axel-delahaye-wl commented 6 years ago

Hello all, for information I have the same issue with CentOS 7.4.1708 x64 on ESXi 6.0 Update 3a (ESXi 6.0 Patch 5 2017-07-11 5572656) and with the open-vm-tools from official centos repo (v10.1.5). But I tested by downgrading with the latest open-vm-tools from CentOS 7.3.1611 and it works (v10.0.5)

Links: (it is possible to get source rpm on http://vault.centos.org/) 7.4 (KO): http://mirror.centos.org/centos-7/7.4.1708/os/x86_64/Packages/open-vm-tools-10.1.5-3.el7.x86_64.rpm 7.3 (OK): http://vault.centos.org/7.3.1611/updates/x86_64/Packages/open-vm-tools-10.0.5-4.el7_3.x86_64.rpm

LapinFou commented 6 years ago

This is the confirmation that there is a regression in Open VM tools. This issue must definitely be fixed!!

Obiwanbde commented 6 years ago

Another test; While I was testing Meltdown patches, I have updated kernel to kernel-3.10.0-693.11.6.el7.x86_64 and situation is the same.

I have tried another action to update kernel to 4.14.13-1.el7.elrepo.x86_64 for CentOS 7.4 on ESXi 6.0U2, and there is no problem with this kernel.

So, it is related with Kernel, I agree with you, it seems this needs to be fixed in Open VM tools.

LapinFou commented 6 years ago

It seems, they don't care... 😠

jonathanvmw commented 6 years ago

@lapinFou No, we care, and my apologies for not reporting back in a more timely manner. However, as I mentioned earlier, filing a support request is likely to improve response times.

@axel-delahey-wl Thanks for reporting your findings. I reproduced them and was able to identify the code change in VMTools responsible for the difference in behavior between VMTools 10.0.5 and 10.1.5 when CentOS 7.4 is running on ESXi 5.5 or 6.0. However, it’s not simply a matter of reverting the change, as the change appears to be appropriate when the guest is running on newer versions of vSphere. Further investigation is needed.

@marco2bb2 A belated thank you for pointing out my error. CentOS 7.x, including 7.4, is supported.

@obiwanbde I will try to get you a better answer as to the meaning of the warning you are getting.

LapinFou commented 6 years ago

@jonathanvmw OK. Good to hear it. 😄 I should get my official VMWare account next week. I will fill up a ticket. In the mean time, we upgraded our hypervisor to ESXi 6.5 (we moved from SAS to SSD). So, it means even if I open a ticket, I'm not concern anymore by this issue.

fchiacchiaretta commented 6 years ago

I have the same problem on ESXi 5.5U3g build 7504623. I also have a FreeBSD 11 VM (pfSense) with open-vm-tools 10.1.0 which faces the same problem:

The configured guest OS (FreeBSD (64-bit)) for this virtual machine does not match the guest that is currently running (FreeBSD 11.1-RELEASE-p6). You should specify the correct guest OS to allow for guest-specific optimizations.

I noticed that in both CentOS and FreeBSD the reported guest OS contains the output of uname -sr:

For CentOS 7.4 - open-vm-tools 10.1.5:

[root@hostname ~]# uname -sr
Linux 3.10.0-693.11.6.el7.x86_64

FreeBSD 11 - open-vm-tools 10.1.0

[2.4.2-RELEASE][admin@hostname]/root: uname -sr
FreeBSD 11.1-RELEASE-p6

For FreeBSD, this is exactly the reported guest OS.

I can also confirm that on CentOS 7.4 - Linux 3.10.0-693.11.6.el7.x86_64 with open-vm-tools 10.0.5 guest OS is reported correctly.

LapinFou commented 6 years ago

@jonathanvmw Finally, I've been able to open a ticket in the VMWare support service. FYI, here is the ticket number: 18689182501

LapinFou commented 6 years ago

VMWare just phoned me. In fact, since the licenses have been provided by HPE, I must open a ticket through HP, not through VMWare. I will keep you in touch when it is done.

Obiwanbde commented 6 years ago

@jonathanvmw

@Obiwanbde I will try to get you a better answer as to the meaning of the warning you are getting.

Hi, any news ?

jonathanvmw commented 6 years ago

@Obiwanbde

Sorry, no update yet. Working on it.

joelafeldmanvmw commented 6 years ago

Hi folks, I think I may be on this thread by accident?

From: jonathanvmw [mailto:notifications@github.com] Sent: Wednesday, January 31, 2018 11:13 AM To: vmware/open-vm-tools open-vm-tools@noreply.github.com Cc: Joel Feldman jfeldman@vmware.com; Mention mention@noreply.github.com Subject: Re: [vmware/open-vm-tools] Centos 7.4 x64 not correctly recognised on ESXi 5.5U3 (#197)

@Obiwanbdehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_obiwanbde&d=DwMCaQ&c=uilaK90D4TOVoH58JNXRgQ&r=uq1u4H2ZOfpu2_NijP6YiLPBC7ZQevAX-f4tO1GMeZM&m=QjhFz-LRGRzZdHIVGtKrIpcs0qLHo3oA4IcHukSf4P0&s=pBkbp-bg4iYQ2iGhPPHJ7BTzK4iCjh_OZbW_2opmV1k&e=

Sorry, no update yet. Working on it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_vmware_open-2Dvm-2Dtools_issues_197-23issuecomment-2D361981851&d=DwMCaQ&c=uilaK90D4TOVoH58JNXRgQ&r=uq1u4H2ZOfpu2_NijP6YiLPBC7ZQevAX-f4tO1GMeZM&m=QjhFz-LRGRzZdHIVGtKrIpcs0qLHo3oA4IcHukSf4P0&s=K3ZfO7XaLbplYCofG_FSVkXGR9cr_2bZzzB8c_R1ZBQ&e=, or mute the threadhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_Afb7FXYcx27Gb6k2znsVLdu30VHLIvbqks5tQJCMgaJpZM4PwvrL&d=DwMCaQ&c=uilaK90D4TOVoH58JNXRgQ&r=uq1u4H2ZOfpu2_NijP6YiLPBC7ZQevAX-f4tO1GMeZM&m=QjhFz-LRGRzZdHIVGtKrIpcs0qLHo3oA4IcHukSf4P0&s=RKFs56pgWH1egHYjfoGchTlRvpDvzVu8Le1Rb5mrllg&e=.

LapinFou commented 6 years ago

My mistake! Github auto-completion... @joelafeldmanvmw instead @jonathanvmw

jonathanvmw commented 6 years ago

@Obiwanbde

Re:

It always says that :
"The configured guest OS (CentOS 4/5/6/7 (64-bit)) for this virtual machine does not match the guest that is currently running (Linux 3.10.0-693.el7.x86_64 CentOS Linux release 7.4.1708 (Core) ). You should specify the correct guest OS to allow for guest-specific optimizations."

Where do you see this message? Thanks.

Obiwanbde commented 6 years ago

@jonathanvmw

Hi,

I see on vSphere host web console, virtual machine details screen. It comes some minutes after booting the VM.

image