vmware / vsphere-automation-sdk-python

Python samples, language bindings, and API reference documentation for vSphere, VMC, and NSX-T using the VMware REST API
MIT License
739 stars 308 forks source link

Failed to import a CA certificate into vCenter Server #384

Closed VedaNiks closed 1 year ago

VedaNiks commented 1 year ago

Describe the bug

I am creating a online depot on vLCM using this API. This call fails if the URL for the depot has HTTPS in it because of certificate trust issues.

To solve that, I am using the code from samples/vsphere/vcenter/certificatemanagement/trusted_root_chains_create.py to import the certificate into vCenter Server.

cert_chain = cert.encode(encoding='utf-8').decode('unicode_escape').split(',') x509_cert_chain = X509CertChain(cert_chain=cert_chain) trusted_root_chains = TrustedRootChains(stub_config) cert_chain_spec = trusted_root_chains.CreateSpec(cert_chain=x509_cert_chain) chain_id = trusted_root_chains.create(cert_chain_spec)

This works fine for a self-signed certificate and depot is created successfully. The problem occurs in case of CA certificate. I am not able to import a CA certificate.

Are there certain requirements the CA certificate needs to fulfil? What can I do to solve this issue? I've tried using vSphere Client UI to import the certificate but encountered the same error. I see below error in /var/log/vmware/certificatemanagement after calling the API:

2023-06-05T04:32:28.759Z [tomcat-exec-7 [] ERROR com.vmware.certificatemanagement.vapi.impl.TrustedRootChainProviderImpl opId=] Exception was thrown while executing create: java.lang.Exception: create trusted root chain failed : Certificate bearing subject CN=fqdn-of-online-depot-machine,OU=IT,O=NGE,L=Chicago,ST=Illinois,C=US is not a valid CA certificate. Please retry with a valid certificate chain at com.vmware.certificatemanagement.impl.trustedroots.TrustedRootsChainCreator.createTrustedChain(TrustedRootsChainCreator.java:115) ~[service-0.0.1-SNAPSHOT.jar:?] at com.vmware.certificatemanagement.vapi.impl.TrustedRootChainProviderImpl.create(TrustedRootChainProviderImpl.java:148) [service-0.0.1-SNAPSHOT.jar:?] at com.vmware.vcenter.certificate_management.vcenter.TrustedRootChainsApiInterface$CreateApiMethod.doInvoke(TrustedRootChainsApiInterface.java:62) [libcom_vmware_vcenter_java.jar:?] at com.vmware.vapi.internal.bindings.ApiMethodSkeleton.invoke(ApiMethodSkeleton.java:232) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.ApiMethodBasedApiInterface.invoke(ApiMethodBasedApiInterface.java:86) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.local.LocalProvider.invokeMethodInt(LocalProvider.java:399) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.local.LocalProvider.invoke(LocalProvider.java:270) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.introspection.ErrorAugmentingFilter.invoke(ErrorAugmentingFilter.java:73) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.authz.impl.AuthorizationFilter.invoke(AuthorizationFilter.java:232) [vapi-authz-2.100.0.jar:?] at com.vmware.vapi.provider.introspection.ErrorAugmentingFilter.invoke(ErrorAugmentingFilter.java:73) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.security.AuthenticationFilter$1.setResult(AuthenticationFilter.java:181) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.security.AuthenticationFilter$1.setResult(AuthenticationFilter.java:165) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.cis.authn.SamlTokenAuthnHandler.authenticate(SamlTokenAuthnHandler.java:61) [vapi-authn-2.100.0.jar:?] at com.vmware.vapi.security.AuthenticationFilter.invoke(AuthenticationFilter.java:164) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.msg.json.JsonServerConnection.processApiRequest(JsonServerConnection.java:396) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.msg.json.JsonServerConnection.requestReceived(JsonServerConnection.java:229) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.rpc.http.impl.HttpStreamingServlet.doPostImpl(HttpStreamingServlet.java:119) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.rpc.http.impl.HttpStreamingServlet.doPost(HttpStreamingServlet.java:88) [vapi-runtime-2.100.0.jar:?] at javax.servlet.http.HttpServlet.service(HttpServlet.java:681) [tomcat-embed-core-8.5.78.jar:8.5.78] at javax.servlet.http.HttpServlet.service(HttpServlet.java:764) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:367) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:639) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:882) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1647) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.78.jar:8.5.78] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_321] 2023-06-05T04:32:28.759Z [tomcat-exec-7 [] WARN com.vmware.vapi.internal.bindings.ApiMethodSkeleton opId=] Implementation method reported unexpected exception: com.vmware.vapi.std.errors.Error com.vmware.vapi.std.errors.Error: Error (com.vmware.vapi.std.errors.error) => { messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => { id = com.vmware.certificatemanagement.error, defaultMessage = Internal Server Error (create trusted root chain failed : Certificate bearing subject CN=fqdn-of-online-depot-machine,OU=IT,O=NGE,L=Chicago,ST=Illinois,C=US is not a valid CA certificate. Please retry with a valid certificate chain), args = [create trusted root chain failed : Certificate bearing subject CN=fqdn-of-online-depot-machine,OU=IT,O=NGE,L=Chicago,ST=Illinois,C=US is not a valid CA certificate. Please retry with a valid certificate chain], params = , localized = }], data = , errorType = ERROR } at com.vmware.certificatemanagement.vapi.impl.TrustedRootChainProviderImpl.create(TrustedRootChainProviderImpl.java:178) [service-0.0.1-SNAPSHOT.jar:?] at com.vmware.vcenter.certificate_management.vcenter.TrustedRootChainsApiInterface$CreateApiMethod.doInvoke(TrustedRootChainsApiInterface.java:62) [libcom_vmware_vcenter_java.jar:?] at com.vmware.vapi.internal.bindings.ApiMethodSkeleton.invoke(ApiMethodSkeleton.java:232) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.ApiMethodBasedApiInterface.invoke(ApiMethodBasedApiInterface.java:86) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.local.LocalProvider.invokeMethodInt(LocalProvider.java:399) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.local.LocalProvider.invoke(LocalProvider.java:270) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.provider.introspection.ErrorAugmentingFilter.invoke(ErrorAugmentingFilter.java:73) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.authz.impl.AuthorizationFilter.invoke(AuthorizationFilter.java:232) [vapi-authz-2.100.0.jar:?] at com.vmware.vapi.provider.introspection.ErrorAugmentingFilter.invoke(ErrorAugmentingFilter.java:73) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.security.AuthenticationFilter$1.setResult(AuthenticationFilter.java:181) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.security.AuthenticationFilter$1.setResult(AuthenticationFilter.java:165) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.cis.authn.SamlTokenAuthnHandler.authenticate(SamlTokenAuthnHandler.java:61) [vapi-authn-2.100.0.jar:?] at com.vmware.vapi.security.AuthenticationFilter.invoke(AuthenticationFilter.java:164) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.msg.json.JsonServerConnection.processApiRequest(JsonServerConnection.java:396) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.msg.json.JsonServerConnection.requestReceived(JsonServerConnection.java:229) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.rpc.http.impl.HttpStreamingServlet.doPostImpl(HttpStreamingServlet.java:119) [vapi-runtime-2.100.0.jar:?] at com.vmware.vapi.protocol.server.rpc.http.impl.HttpStreamingServlet.doPost(HttpStreamingServlet.java:88) [vapi-runtime-2.100.0.jar:?] at javax.servlet.http.HttpServlet.service(HttpServlet.java:681) [tomcat-embed-core-8.5.78.jar:8.5.78] at javax.servlet.http.HttpServlet.service(HttpServlet.java:764) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:197) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:543) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:367) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:639) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:882) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1647) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659) [tomcat-embed-core-8.5.78.jar:8.5.78] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.78.jar:8.5.78] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_321]

Reproduction steps

  1. Generate a CA certificate from internal CA or public CA.
  2. Go to vSphere Client Menu > Administration > Certificates > Certificate Management > Trusted Root Certificates > ADD
  3. Import fails with error Error occurred while adding trusted root certificates: com.vmware.vapi.std.errors.Error, create trusted root chain failed : Certificate bearing subject CN=fqdn-of-online-depot-machine, OU=IT,O=NGE,L=Chicago,ST=Illinois,C=US is not a valid CA certificate. Please retry with a valid certificate chain.

Expected behavior

The CA certificate should get imported similar to self signed SSL certificate.

Additional context

No response

kunal-pmj commented 1 year ago

you need to add the complete chain. Make sure you have Root CA, Intermediate CA(s) and the issuing cert.

The certificates you are adding using the api do not complete the chain.

https://en.wikipedia.org/wiki/Chain_of_trust

VedaNiks commented 1 year ago

Hello @kunal-pmj, Thank you for you response. I've tried to import the complete chain but failed with same error. Can you please provide me some guidance? I've tried below things:

1] I am fetching the certificate from the online depot VM and trying to import it:

def get_ssl_cert(address, port=443):
    server_address = (address, port)

    # Retrieve the server certificate in PEM format
    cert = ssl.get_server_certificate(server_address)
    return cert

def import_ssl_cert_into_vcenter():
    cert = get_ssl_cert('online-depot-vm-ip')
    cert_chain = cert.encode(encoding='utf-8').decode('unicode_escape').split(',')
    x509_cert_chain = X509CertChain(cert_chain=cert_chain)
    trusted_root_chains = TrustedRootChains(stub_config)
    cert_chain_spec = trusted_root_chains.CreateSpec(cert_chain=x509_cert_chain)
    chain_id = trusted_root_chains.create(cert_chain_spec)

Here as per you comment, the import is failing because I am adding only leaf cert and not the complete chain. So, I have tried below: I got the certificate in PEM format which had ROOT, INTERMEDIATE and LEAF certificate.

i] I tried to import this certificate using below code:

def import_full_certificate():
    file_path = 'path-to-certificate'
    with open(file_path, "r") as my_cert_file:
        cert_pem = my_cert_file.read()
        cert_chain = cert_pem.encode(encoding='utf-8').decode('unicode_escape').split(',')

        x509_cert_chain = X509CertChain(cert_chain=cert_chain)
        cert_chain = TrustedRootChains.CreateSpec(cert_chain=x509_cert_chain)

        print('The alias of the certificate chain successfully imported into vCenter listed below ')
        print(vsphere_client.vcenter.certificate_management.vcenter.TrustedRootChains.create(cert_chain))

This failed in same error. Certificate bearing subject CN=fqdn-of-online-depot-machine,OU=IT,O=NGE,L=Chicago,ST=Illinois,C=US is not a valid CA certificate. Please retry with a valid certificate chain],

ii] I tried to import the certificates in below sequence ROOT > INTERMEDIATE > LEAF :

def divide_and_import_certificate():
    file_path = 'path-to-certificate'
    certs = pem.parse_file(file_path)  # using pem module
    str_cert_list = []
    for pem_certificates in certs:
        str_cert_list.append(str(pem_certificates))

    # after splitting root cert was last
    for str_cert in reversed(str_cert_list): 
        cert_chain = str_cert.encode(encoding='utf-8').decode('unicode_escape').split(',')
        x509_cert_chain = X509CertChain(cert_chain=cert_chain)
        cert_chain = TrustedRootChains.CreateSpec(cert_chain=x509_cert_chain)
        print(vsphere_client.vcenter.certificate_management.vcenter.TrustedRootChains.create(cert_chain))

In this case ROOT and INTERMEDIATE certificate was successfully imported but it failed with same error in case of LEAF certificate.

I am not sure what I am doing wrong. Can you please guide me? Is this possible that since the CA is internal we have to do some additional steps? I've downloaded [full chain using openssl command] some SSL certificates from sites like www.nvidia.com, www.hp.com and tried to import them but failed.

kunal-pmj commented 1 year ago

Kindly verify if you are missing some intermediate certificate.

There are online tools available which decode your SSL certificate to check if you are missing an intermediate certificate(s).

Also check if the PEM format can be converted to X509. If the certificates are valid they should get converted.

VedaNiks commented 1 year ago

Hi @kunal-pmj, I have checked that certificate chain is complete using this https://tools.keycdn.com/ssl.

I am attaching a certificate I generated by creating a local root CA and intermediate CA. I've tested it on the above site. Can you please check and let me know why I can't import this certificate? cert.zip

Also, can you verify that cert import is necessary before creating a online depot in vLCM?

Thanks in advance!

kunal-pmj commented 1 year ago

@VedaNiks

The extension "keyUsage": must have the "Certificate Sign" for all the certs in the chain

You have it only for the CA :

"keyUsage": "Digital Signature, Certificate Sign, CRL Sign"

You can see it in the https://tools.keycdn.com/ssl as well with your certs.

kunal-pmj commented 1 year ago

Intermediate or selfsigned rooted, they must be valid CA certs else will be rejected. Key Encipherment is not the usage of a CA cert its an enduser cert usage and should not and can not be aded to Trusted Roots