The plugin doesn't display VCHs and containers after updating to 1.4.4

[qxmips]

For stories, please include the information below:

User Statement:

after updating VIC from 1.4.3 to 1.4.4 the web pluging doesn't show VHC endpoints and containers screenshots are attached.

Details: VMware VirtualCenter 6.7.0 build-9433931 ESXi, 6.7.0, 8169922 there are two PSCs behind an A10 load-balancer for this vcenter and a linked vcenter on another site with one PCS .

Acceptance Criteria: Specific deliverable and actionable items that the story needs to deliver in order to be considered complete, the more detail here the more accurate the implementation will be.

For bug reports, please include the information below:

VIC-UI version:

pluging -v1.4.4.1405 VIC Version vic01-san.ztelco.local-v1.4.4 Platform details:

Which platform are you using? (Windows, Mac OS, Linux, etc)

Browser details:

Which browser are you using? (IE, Chrome, Firefox, etc)

Additional details:

If applicable please include the following:

Error messages. (In the UI or from the browser's developer tools)

Request/response, including headers (from the browser's developer tools).

Steps to reproduce:

Actual behavior: the plugin doesnt show VCHs and containers

Expected behavior: plugin to show VCHs and containers

Logs:

[2018-12-13T04:29:58.239Z] [INFO ] http-nio-5090-exec-15 70063839 105084 200213 c.v.v.mvc.objectnavigator.ObjectNavigatorNavigationManagerImpl handleNavigationRequest(). Could not interpr et navigation request. targetID=com.vmware.vic.objectView.summary ,context=urn:vic:vic:Root:vic%252Fvic-root [2018-12-13T04:30:04.103Z] [WARN ] data-service-pool-791 70063841 105084 200213 c.v.v.client.iconslabels.IconsLabelsPropertyProviderAdapter No icon property found [2018-12-13T04:31:38.526Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vsphere.client.security.sso.SsoTokenLifetimeManager Beginning renewal of token for sessionId 105 084, clientId 200213 [2018-12-13T04:31:38.528Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vise.vim.security.sso.impl.SsoLsLocatorImpl Fetching SSO endpoints from LS [2018-12-13T04:31:38.628Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vise.vim.security.sso.impl.SsoLsLocatorImpl Retrieved locations of services from LS at h ttps://psc-san.ztelco.local/lookupservice/sdk in 101 milliseconds: {vsphere.local(2e9f6280-d3c2-448f-840f-363b8d8687db@L)=adminAppUrl: https://psc-san.ztelco.local/sso-adminserver/sdk/vsphere.local, stsUrl: https://psc-san.ztelco.local/sts/STSService/vsphere.local , metadataUrl: https://psc-san.ztelco.local/websso/SAML2/Metadata/vsphere.local, vmodlVersion: interface com.vmware.vim.binding.sso.version.version3_5} [2018-12-13T04:31:38.628Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vise.vim.security.sso.impl.SsoUtilInternal Renewing token _324c5cf0-ba8d-4568-89ab-6300 a1e1be09 using https://psc-san.ztelco.local/sts/STSService/vsphere.local [2018-12-13T04:31:38.646Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vim.sso.client.impl.SiteAffinityServiceDiscovery Site affinity is disabled [2018-12-13T04:31:38.777Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.identity.token.impl.SamlTokenImpl SAML token for SubjectNameId [value=Administ rator@VSPHERE.LOCAL, format=http://schemas.xmlsoap.org/claims/UPN] successfully parsed from Element [2018-12-13T04:31:38.778Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vim.sso.client.impl.SecurityTokenServiceImpl Successfully renewed token for user: {Name: Administrator, Domain: VSPHERE.LOCAL} [2018-12-13T04:31:38.778Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vise.vim.security.sso.AsyncTokenManager Updated the token of domain vsphere.local(2e 9f6280-d3c2-448f-840f-363b8d8687db@L). Token's expiration time: Thu Dec 13 12:31:38 UTC 2018 [2018-12-13T04:31:38.778Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vsphere.client.security.sso.SsoTokenLifetimeManager Token renewed for sessionId 105084, clientId

200213. Used STS of domain vsphere.local(2e9f6280-d3c2-448f-840f-363b8d8687db@L) for the renewal. Token expiration time: Thu Dec 13 12:31:38 UTC 2018 [2018-12-13T04:31:38.779Z] [INFO ] oTokenRenewalExecutor-132985 ######## 105084 200213 com.vmware.vsphere.client.security.sso.SsoTokenLifetimeManager sessionId 105084, clientId 200213, Token exp iration time: Thu Dec 13 12:31:38 UTC 2018 Token renewal scheduled approximately for: Thu Dec 13 12:26:38 UTC 2018 [2018-12-13T04:31:51.259Z] [INFO ] health-status-72 com.vmware.vise.vim.cm.healthstatus.AppServerHealthService Memory usage: used=473,535,448; max=1,126,170,624; percentage=42.0 4828628170645%. Status: GREEN

---

/storage/log/vmware/vsphere-ui/logs ]# tail -f vsphere_client_virgo.log at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingTask.call(ExecutorUtil.java:1234) at com.vmware.vise.data.query.impl.DataServiceThreadPoolDecorator$1.call(DataServiceThreadPoolDecorator.java:174) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

[2018-12-13T04:34:48.479Z] [INFO ] http-nio-5090-exec-30 70063850 105084 200213 c.v.v.mvc.objectnavigator.ObjectNavigatorNavigationManagerImpl handleNavigationRequest(). Could not interpret navigation request. targetID=com.vmware.vic.customtab-container ,context=urn:vic:vic:Root:vic%252Fvic-root [2018-12-13T04:34:51.260Z] [INFO ] health-status-72 com.vmware.vise.vim.cm.healthstatus.AppServerHealthService Memory usage: used=441,937,080; max=1,126,170,624; percentage=39.242462072958496%. Status: GREEN [2018-12-13T04:34:51.260Z] [INFO ] health-status-72 c.v.v.v.cm.HealthStatusRequestHandler$HealthStatusCollectorTask Determined health status 'GREEN' in 0 ms [2018-12-13T04:36:42.730Z] [ERROR] http-nio-5090-exec-30 70063858 105084 200213 com.vmware.vise.mvc.objectnavigator.ObjectNavigatorManagerImpl Incorrect extension detected for vise.navigator.nodespecs extension point. There are multiple extensions with the same navigationTargetUid: com.vmware.vic.objectView_collection.domainView, but from different parent chains. [2018-12-13T04:36:42.730Z] [ERROR] http-nio-5090-exec-30 70063858 105084 200213 com.vmware.vise.mvc.objectnavigator.ObjectNavigatorManagerImpl Incorrect extension detected for vise.navigator.nodespecs extension point. There are multiple extensions with the same navigationTargetUid: com.vmware.vic.objectView_collection.domainView, but from different parent chains. [2018-12-13T04:36:42.735Z] [ERROR] http-nio-5090-exec-15 70063859 105084 200213 com.vmware.vise.mvc.objectnavigator.ObjectNavigatorManagerImpl Incorrect extension detected for vise.navigator.nodespecs extension point. There are multiple extensions with the same navigationTargetUid: com.vmware.vic.objectView_collection.domainView, but from different parent chains. [2018-12-13T04:36:42.735Z] [ERROR] http-nio-5090-exec-15 70063859 105084 200213 com.vmware.vise.mvc.objectnavigator.ObjectNavigatorManagerImpl Incorrect extension detected for vise.navigator.nodespecs extension point. There are multiple extensions with the same navigationTargetUid: com.vmware.vic.objectView_collection.domainView, but from different parent chains. [2018-12-13T04:36:45.392Z] [WARN ] data-service-pool-791 70063863 105084 200213 c.v.v.client.iconslabels.IconsLabelsPropertyProviderAdapter No icon property found [2018-12-13T04:37:12.068Z] [ERROR] http-nio-5090-exec-13 70063865 105084 200213 c.v.v.mvc.objectnavigator.ObjectNavigatorNavigationManagerImpl Error in interpreting request, interpreter=com.vmware.vise.mvc.objectnavigator.interpreter.ObjectNodeNavReqInterpreter@46ac4d7 error: com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: A specified parameter was not correct: token Please see the server log to find more detail regarding exact cause of the failure. at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178) at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:124) at com.sun.xml.internal.ws.client.sei.StubHandler.readResponse(StubHandler.java:238) at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:189) at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:276) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:104) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:77) at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:147) at com.sun.proxy.$Proxy898.continueRetrievePropertiesEx(Unknown Source) at com.vmware.vic.PropFetcher.getVicVms(PropFetcher.java:368) at com.vmware.vic.ObjectStore.getRootObject(ObjectStore.java:96) at com.vmware.vic.ObjectStore.getObj(ObjectStore.java:125) at com.vmware.vic.VicUIDataAdapter.processObjectIdentityConstraint(VicUIDataAdapter.java:161) at com.vmware.vic.VicUIDataAdapter.processConstraint(VicUIDataAdapter.java:127) at com.vmware.vic.VicUIDataAdapter.processQuerySpec(VicUIDataAdapter.java:106) at com.vmware.vic.VicUIDataAdapter.getData(VicUIDataAdapter.java:86) at com.vmware.vise.data.query.impl.DataAdapterUtil.getDataFromAdapter(DataAdapterUtil.java:219) at com.vmware.vise.data.query.impl.DataAdapterUtil.getResultFromAdapter(DataAdapterUtil.java:176) at com.vmware.vise.data.query.impl.DataAdapterUtil.access$000(DataAdapterUtil.java:43) at com.vmware.vise.data.query.impl.DataAdapterUtil$1.call(DataAdapterUtil.java:82) at com.vmware.vise.data.query.impl.DataAdapterUtil$1.call(DataAdapterUtil.java:79) at com.vmware.vise.util.concurrent.ExecutorUtil$2.call(ExecutorUtil.java:773) at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingTask.call(ExecutorUtil.java:1234) at com.vmware.vise.data.query.impl.DataServiceThreadPoolDecorator$1.call(DataServiceThreadPoolDecorator.java:174) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

[2018-12-13T04:37:12.069Z] [INFO ] http-nio-5090-exec-13 70063865 105084 200213 c.v.v.mvc.objectnavigator.ObjectNavigatorNavigationManagerImpl handleNavigationRequest(). Could not interpret navigation request. targetID=com.vmware.vic.customtab-vch ,context=urn:vic:vic:Root:vic%252Fvic-root [2018-12-13T04:37:12.484Z] [WARN ] data-service-pool-768 70063867 105084 200213 c.v.v.client.iconslabels.IconsLabelsPropertyProviderAdapter No icon property found [2018-12-13T04:37:12.489Z] [ERROR] http-nio-5090-exec-15 70063867 105084 200213 com.vmware.vise.mvc.exception.GlobalExceptionHandler Exception handled: com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: A specified parameter was not correct: token Please see the server log to find more detail regarding exact cause of the failure. at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178) at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:124) at com.sun.xml.internal.ws.client.sei.StubHandler.readResponse(StubHandler.java:238) at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:189) at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:276) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:104) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:77) at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:147) at com.sun.proxy.$Proxy898.continueRetrievePropertiesEx(Unknown Source) at com.vmware.vic.PropFetcher.getVicVms(PropFetcher.java:368) at com.vmware.vic.ObjectStore.getRootObject(ObjectStore.java:96) at com.vmware.vic.ObjectStore.getObj(ObjectStore.java:125) at com.vmware.vic.VicUIDataAdapter.processObjectIdentityConstraint(VicUIDataAdapter.java:161) at com.vmware.vic.VicUIDataAdapter.processConstraint(VicUIDataAdapter.java:127) at com.vmware.vic.VicUIDataAdapter.processQuerySpec(VicUIDataAdapter.java:106) at com.vmware.vic.VicUIDataAdapter.getData(VicUIDataAdapter.java:86) at com.vmware.vise.data.query.impl.DataAdapterUtil.getDataFromAdapter(DataAdapterUtil.java:219) at com.vmware.vise.data.query.impl.DataAdapterUtil.getResultFromAdapter(DataAdapterUtil.java:176) at com.vmware.vise.data.query.impl.DataAdapterUtil.access$000(DataAdapterUtil.java:43) at com.vmware.vise.data.query.impl.DataAdapterUtil$1.call(DataAdapterUtil.java:82) at com.vmware.vise.data.query.impl.DataAdapterUtil$1.call(DataAdapterUtil.java:79) at com.vmware.vise.util.concurrent.ExecutorUtil$2.call(ExecutorUtil.java:773) at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingTask.call(ExecutorUtil.java:1234) at com.vmware.vise.data.query.impl.DataServiceThreadPoolDecorator$1.call(DataServiceThreadPoolDecorator.java:174) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

[2018-12-13T04:37:17.244Z] [ERROR] extfw-filterer-pool-132998 70063870 105084 200213 com.vmware.vise.util.concurrent.ExecutorUtil A task crashed: com.vmware.vise.extensionfw.impl.Filterer$1@b107e94 com.sun.xml.internal.ws.fault.ServerSOAPFaultException: Client received SOAP Fault from server: A specified parameter was not correct: token Please see the server log to find more detail regarding exact cause of the failure. at com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178) at com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:124) at com.sun.xml.internal.ws.client.sei.StubHandler.readResponse(StubHandler.java:238) at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:189) at com.sun.xml.internal.ws.db.DatabindingImpl.deserializeResponse(DatabindingImpl.java:276) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:104) at com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:77) at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:147) at com.sun.proxy.$Proxy898.continueRetrievePropertiesEx(Unknown Source) at com.vmware.vic.PropFetcher.getVicVms(PropFetcher.java:368) at com.vmware.vic.ObjectStore.getRootObject(ObjectStore.java:96) at com.vmware.vic.ObjectStore.getObj(ObjectStore.java:125) at com.vmware.vic.VicUIDataAdapter.processObjectIdentityConstraint(VicUIDataAdapter.java:161) at com.vmware.vic.VicUIDataAdapter.processConstraint(VicUIDataAdapter.java:127) at com.vmware.vic.VicUIDataAdapter.processQuerySpec(VicUIDataAdapter.java:106) at com.vmware.vic.VicUIDataAdapter.getData(VicUIDataAdapter.java:86) at com.vmware.vise.data.query.impl.DataAdapterUtil.getDataFromAdapter(DataAdapterUtil.java:219) at com.vmware.vise.data.query.impl.DataAdapterUtil.getResultFromAdapter(DataAdapterUtil.java:176) at com.vmware.vise.data.query.impl.DataAdapterUtil.access$000(DataAdapterUtil.java:43) at com.vmware.vise.data.query.impl.DataAdapterUtil$1.call(DataAdapterUtil.java:82) at com.vmware.vise.data.query.impl.DataAdapterUtil$1.call(DataAdapterUtil.java:79) at com.vmware.vise.util.concurrent.ExecutorUtil$2.call(ExecutorUtil.java:773) at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingTask.call(ExecutorUtil.java:1234) at com.vmware.vise.data.query.impl.DataServiceThreadPoolDecorator$1.call(DataServiceThreadPoolDecorator.java:174) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

[renmaosheng]

Meina, could you please take a look? thanks.

[zhoumeina]

OK. I WILL TAKE CARE OF THIS.

[zhoumeina]

Hi @qxmips , do you have a vc environment with this issue which I can login?

[qxmips]

Hi @zhoumeina I think I can arrange a remote access for you. Would a TeamViewer or similar remote access tool work for you?

[zhoumeina]

@qxmips , can you provide me the vc ip?

[qxmips]

@qxmips , can you provide me the vc ip?

it's in an internal network there is no public IP for the VC.

[zhoumeina]

ok. @qxmips , could you please show me the log of vsphere-ui? and open the browser developer console to find if there are any request status code is not 200?

[qxmips]

@zhoumeina Status 500 for Requests URL: https://vcsa-san.ztelco.local/ui/vic/rest/data/list/?targetType=vic:ContainerVm&properties=match,results&sorting=id,asc&filter=&offset=0&maxResultCount=10 https://vcsa-san.ztelco.local/ui/data/properties/urn:vic:vic:Root:vic%25252Fvic-root?properties=name,primaryIconId https://vcsa-san.ztelco.local/ui/vic/rest/data/list/?targetType=vic:VirtualContainerHostVm&properties=match,results&offset=0&maxResultCount=10&

log from a browser.txt vsphere-ui.logs.txt

[zhoumeina]

@qxmips , I tried to upgrade from 1.4.3 to 1.4.4 , everything goes on well. I don't have psc configed in my environment. I think I need your environment to investigate.

[qxmips]

@zhoumeina have you found any clues in the logs? at which directions should I look at.

[qxmips]

is there a way of manually installing (downgrading to) the previous version of the plugin?

[zhoumeina]

what version you want to install? 1.4.3?@qxmips

[zhoumeina]

@qxmips , reinstall 1.4.3 will work.

[zhoumeina]

Let me install a teamviewer quickly.

[zhoumeina]

@qxmips , how can you send me your id?

[zhoumeina]

@qxmips , are you there? How can I contact you? By slack?

[zhoumeina]

I think may be we need to investigate it together

[qxmips]

I think may be we need to investigate it together Sorry just saw you messages. Hi by slack id in vmware.code is @qxmips

[dbarkelew]

@zhoumeina I have a case with the same and have some logs. I put them on bug 2257085.

[dbarkelew]

is there a way of manually installing (downgrading to) the previous version of the plugin?

@qxmips Before going through the effort of attempting a downgrade could you use the vic-machine cli as a temporary workaround?

[qxmips]

@dbarkelew Sure, the problem is only cosmetic for us. Just want to find out if it's a known bug or some misconfiguration in our VC - VIC environment.

[wjun]

@qxmips From the vsphere-ui.log, it looks the html5 web server crashes. Can you try the following command to restart your html5 web server on your vCenter appliance and see if the issue disappear? service-control --stop vsphere-ui && service-control --start vsphere-ui

[qxmips]

@qxmips From the vsphere-ui.log, it looks the html5 web server crashes. Can you try the following command to restart your html5 web server on your vCenter appliance and see if the issue disappear? service-control --stop vsphere-ui && service-control --start vsphere-ui

@wjun Hi , tried that before, anyway restarted again. same thing.

[zhoumeina]

[zhoumeina]

[zhoumeina]

@qxmips , is this the same issue with https://bugzilla.eng.vmware.com/show_bug.cgi?id=2257085

[renmaosheng]

meina, I don't think @qxmips can access the bugzilla. @qxmips please ignore above question, you mentioned the issue is found by upgrading to 1.4.4, could you please confirm all the other configuration like the VC PSC configuration is the same comparing when you are in 1.4.3?

[qxmips]

Hi @renmaosheng . Yes the config for PCS hasn't changed. I had upgraded VIC Appliance from 1.4.3 to 1.4.4 turns out the previous appliance had v1.4.1.1262.zip plugin. so it was v1.4.1.1262 to 1.4.4 plugin update path.

[zhoumeina]

@qxmips , I fixed this issue by pr: #660 which is merged to master. This fix can be get in vic 1.5.2 release. I tested in in the link mode with 2 psc. Not sure it is the exactly same environment with yours. Please feel free to open this if you test 1.5.2 still have this problem.