search

vmware / vic-product

vSphere Integrated Containers enables VMware customers to deliver a production-ready container solution to their developers and DevOps teams.
https://vmware.github.io/vic-product/
Other
177 stars 92 forks source link

Unable to open Admiral page after successful registration #1126

Closed lgayatri closed 6 years ago

lgayatri commented 6 years ago

@mdubya66 @sergiosagu

VCSA: CLOUDVM_VERSION:6.6.3.10000 VIC 1.3

image

vmware-identity-sts-default.log says:

[2017-11-22T10:46:40.667Z pool-2-thread-4                                                           INFO  com.vmware.identity.admin.vlsi.ConfigurationManagementServiceImpl] [User {Name: Administrator, Domain: VSPHERE.LOCAL} with role 'Administrator'] Importing SAML metadata
[2017-11-22T10:46:40.724Z pool-2-thread-4                                                           INFO  com.vmware.identity.idm.client.SAMLImporter] Create or updated Relying Party: https://10.197.37.186:8282/saml/websso/metadata
[2017-11-22T10:46:40.724Z pool-2-thread-4                                                           INFO  com.vmware.identity.admin.vlsi.ConfigurationManagementServiceImpl] Vmodl method ConfigurationManagementService.importSAMLMetadata return value is null
...
[2017-11-22T10:46:41.854Z pool-2-thread-3                                                           INFO  com.vmware.identity.vlsi.RoleBasedAuthorizer] User {Name: Administrator, Domain: VSPHERE.LOCAL} with role 'Administrator' is authorized for method call 'SessionManager.login'
[2017-11-22T10:46:41.854Z pool-2-thread-5                                                           INFO  com.vmware.identity.vlsi.SessionManagerImpl] Login called
[2017-11-22T10:46:41.856Z pool-2-thread-5                                                           INFO  com.vmware.identity.vlsi.SessionManagerImpl] User {Name: Administrator, Domain: VSPHERE.LOCAL} with role 'Administrator' logged in successfully.
[2017-11-22T10:46:41.870Z pool-2-thread-5                                                           INFO  com.vmware.identity.vlsi.RoleBasedAuthorizer] User {Name: Administrator, Domain: VSPHERE.LOCAL} with role 'Administrator' is authorized for method call 'IdentitySourceManagementService.getSslCertificateManager'
[2017-11-22T10:46:41.884Z pool-2-thread-5                                                           INFO  com.vmware.identity.vlsi.RoleBasedAuthorizer] User {Name: Administrator, Domain: VSPHERE.LOCAL} with role 'Administrator' is authorized for method call 'PrincipalManagementService.deleteLocalPrincipal'
[2017-11-22T10:46:41.885Z pool-2-thread-4                                                           INFO  com.vmware.identity.admin.vlsi.PrincipalManagementServiceImpl] [User {Name: Administrator, Domain: VSPHERE.LOCAL} with role 'Administrator'] Deleting principal 'vic-cloud-admin'
[2017-11-22T10:46:41.892Z pool-2-thread-4 vsphere.local        10c37fd3-398e-4126-8b9f-10bb5b6f75d7 ERROR com.vmware.identity.idm.server.IdentityManager] Failed to delete principalName [vic-cloud-admin] in tenant [vsphere.local]
[2017-11-22T10:46:41.892Z pool-2-thread-4 vsphere.local        10c37fd3-398e-4126-8b9f-10bb5b6f75d7 ERROR com.vmware.identity.idm.server.ServerUtils] Exception 'com.vmware.identity.idm.InvalidPrincipalException: Principal name: vic-cloud-admin@vsphere.local doesn't exist.'
com.vmware.identity.idm.InvalidPrincipalException: Principal name: vic-cloud-admin@vsphere.local doesn't exist.
        at com.vmware.identity.idm.server.provider.vmwdirectory.VMwareDirectoryProvider.deletePrincipal(VMwareDirectoryProvider.java:4519) ~[vmware-identity-idm-server-7.0.0.jar:?]
        at com.vmware.identity.idm.server.IdentityManager.deletePrincipal(IdentityManager.java:5947) ~[vmware-identity-idm-server-7.0.0.jar:?]
        at com.vmware.identity.idm.server.IdentityManager.deletePrincipal(IdentityManager.java:10619) [vmware-identity-idm-server-7.0.0.jar:?]
        at com.vmware.identity.idm.client.CasIdmClient.deletePrincipal(CasIdmClient.java:2701) [vmware-identity-idm-client-7.0.0.jar:?]
        at com.vmware.identity.admin.server.ims.impl.PrincipalManagementImpl.deleteLocalPrincipal(PrincipalManagementImpl.java:1588) [sso-adminserver-7.0.0.jar:?]
        at com.vmware.identity.admin.vlsi.PrincipalManagementServiceImpl$4.call(PrincipalManagementServiceImpl.java:152) [sso-adminserver-7.0.0.jar:?]
        at com.vmware.identity.admin.vlsi.PrincipalManagementServiceImpl$4.call(PrincipalManagementServiceImpl.java:140) [sso-adminserver-7.0.0.jar:?]
        at com.vmware.identity.admin.vlsi.util.VmodlEnhancer.invokeVmodlMethod(VmodlEnhancer.java:160) [sso-adminserver-7.0.0.jar:?]
        at com.vmware.identity.admin.vlsi.PrincipalManagementServiceImpl.deleteLocalPrincipal(PrincipalManagementServiceImpl.java:140) [sso-adminserver-7.0.0.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_151]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_151]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_151]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_151]
        at com.vmware.vim.vmomi.server.impl.InvocationTask.run(InvocationTask.java:65) [vlsi-server-7.0.0.jar:?]

We have the required user already on VCSA: image

Expected Admiral to open successfully. Live setup available upon request.

mdubya66 commented 6 years ago

@sergiosagu @martin-borisov is this an admiral issue?

andrewtchin commented 6 years ago

@lgayatri When opening issues, please include the OVA name such as vic-dev-2571-v1.3.0-dev1-200-ga9048f6.ova because VIC 1.3 does not currently exist and it is very difficult to root cause this if we don't know the version of the components that are running.

lgayatri commented 6 years ago

@andrewtchin , sure- will do. Here is the build: vic-dev-2419-v1.3.0-dev1-178-gfbdb6d5.ova

sergiosagu commented 6 years ago

The actual issue seems to be a failure during the registration, or in other words, the registration wasn't successful... or wasn't completely successful.

The vmware-identity-sts-default.log logs here are misleading since that's probably an expected error message in clean environments. The registration process, before trying to create the vic-cloud-admin default user, first it tries to delete it in case it already exists (and since it doesn't exist, that's the error displayed in the vmware-identity-sts-default.log file).

@lgayatri - where did you see that the registration was successful? can you attach the registration logs?

lgayatri commented 6 years ago

@sergiosagu I dont have the setup , but I am sure that the registration succeeded for UI. As I always open the management portal from web page on 9443 . Is there a separate log for registration? I usually check journalctl

andrewtchin commented 6 years ago

@lgayatri The registration log from the Getting Started Page is in journalctl -u fileserver Also please always include the host/credentials either in the issue or in the vic-product-standup Slack channel with a reference the bug number so that there isn't a delay in triaging

lgayatri commented 6 years ago

@andrewtchin , here is the repro.

VCSA: 6.6.3.10000 VIC 1.3 : vic-v1.3.0-rc4-2870-f8cc7317.ova

PSC registration is successful

image

journalctl -u fileserver

Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=info msg="server URL: cloud02-w3.stls.local\n"
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=debug msg="Accepting host \"cloud02-w3.stls.local\" thumbprint 03:C7:7C:AE:4C:6C:20:40:8F:A
B:C3:43:99:B9:7B:46:DA:BD:0D:21"
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=debug msg="Creating VMOMI session with thumbprint 03:C7:7C:AE:4C:6C:20:40:8F:AB:C3:43:99:B9
:7B:46:DA:BD:0D:21"
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=debug msg="Session Environment Info: " API Type=VirtualCenter API Version=6.7 Build=7362297
 Name="VMware vCenter Server" OS Type=linux-x64 Product ID=vpx UUID=87036cae-8d5e-4639-867b-e0b123c4b99b Vendor="VMware, Inc." Version=6.6.3
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=debug msg="vSphere resource cache populating..."
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=debug msg="Error count populating vSphere cache: (5)"
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=debug msg="new validator Session.Populate: Failure finding dc (): default datacenter resolv
es to multiple instances, please specify\nFailure finding cluster (): please specify a datacenter\nFailure finding ds (): please specify a datacenter\nFailure finding host (): please specify a data
center\nFailure finding pool (): please specify a datacenter"
Dec 19 06:38:39 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:39Z" level=info msg="Validation succeeded"
Dec 19 06:38:40 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:40Z" level=debug msg="successfully attached the product tag"
Dec 19 06:38:40 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:40Z" level=info msg="User domain: vsphere.local PSC domain: vsphere.local. Using vsphere.local"
Dec 19 06:38:40 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:40Z" level=info msg="vCenter user: administrator@vsphere.local"
Dec 19 06:38:40 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:40Z" level=info msg="PSC instance: w3-stras-sso01.stls.local"
Dec 19 06:38:40 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:40Z" level=info msg="PSC domain: vsphere.local"
Dec 19 06:38:40 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:40Z" level=info msg="PSC Out of the box users. CreateUsers: True, FoundCreateUsers: true, Prefix: vic"
Dec 19 06:38:43 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:43Z" level=info msg="Successfully registered harbor with PSC"
Dec 19 06:38:46 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:46Z" level=info msg="Successfully registered engine with PSC"
Dec 19 06:38:54 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:54Z" level=info msg="Successfully registered admiral with PSC"
Dec 19 06:38:54 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:54Z" level=info msg="render: html/index.html"
Dec 19 06:38:54 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:54Z" level=info msg="render: /opt/vmware/fileserver/html/index.html"
Dec 19 06:38:55 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:55Z" level=info msg="render: html/index.html"
Dec 19 06:38:55 vic-st-h2-189.eng.vmware.com start_fileserver.sh[1467]: time="2017-12-19T06:38:55Z" level=info msg="render: /opt/vmware/fileserver/html/index.html"

Admiral page does not open

image

admiral log says:

[359][I][2017-12-19T06:50:05.685Z][11][8282/][lambda$schedulePeriodicCertificatesReload$1][Host https://vic-st-h2-189.eng.vmware.com:8282/: reloading all certificates]
[360][W][2017-12-19T06:53:23.928Z][125][8282/auth/psc/callback][redirectToSamlSso][Could not generate redirect URL: java.lang.IllegalStateException: SsoManager has not been initialized
        at com.vmware.admiral.auth.idm.psc.saml.util.SamlManager.getInstance(SamlManager.java:100)
        at com.vmware.admiral.auth.idm.psc.saml.sso.authentication.SamlRequestSender.<init>(SamlRequestSender.java:45)
        at com.vmware.admiral.auth.idm.psc.saml.util.SsoUriGenerator.generateRedirectUrl(SsoUriGenerator.java:47)
        at com.vmware.admiral.auth.idm.psc.service.PscAuthenticationService.redirectToSamlSso(PscAuthenticationService.java:457)
        at com.vmware.admiral.auth.idm.psc.service.PscAuthenticationService.redirectToSso(PscAuthenticationService.java:451)
        at com.vmware.admiral.auth.idm.psc.service.PscAuthenticationService.handleGet(PscAuthenticationService.java:435)
        at com.vmware.xenon.common.StatelessService.handleRequest(StatelessService.java:120)
        at com.vmware.xenon.common.StatelessService.handleRequest(StatelessService.java:103)
        at com.vmware.xenon.common.ServiceHost.lambda$queueOrScheduleRequestInternal$44(ServiceHost.java:4292)
        at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
]

Will ping the setup details on standup channel.

@mdubya66 - Lets target this bug for 1.4 given the timelines.

lgayatri commented 6 years ago

VCSA has external PSC with F5 Load balancer (PSCs are behind LB)

lgayatri commented 6 years ago

I tried on embedded VCSA with version 6.6.3 and still see the same issue that despite successful VCSA registration, admiral does not open.

andrewtchin commented 6 years ago

Per conversation on slack it looks like this is an Admiral issue with PSC on 6.6.3

andrewtchin commented 6 years ago

@lgayatri Is there an issue open in Admiral to track this? If so, I will close this. If not, I will move it to Admiral repo.

sergiosagu commented 6 years ago

@andrewtchin / @lgayatri - We have a task in our Jira to track it (VBV-1791). Feel free to close this issue. cc @lazarin

martin-borisov commented 6 years ago

This is a feature that we are tracking in jira. Closing this one.