confluentinc / cp-demo

Confluent Platform Demo including Apache Kafka, ksqlDB, Control Center, Schema Registry, Security, Schema Linking, and Cluster Linking
Apache License 2.0
38 stars 322 forks source link

Error when running ./scripts/start.sh. Related to SSL certificates verification behind VPN #381

Closed igal-ore-ssq closed 3 years ago

igal-ore-ssq commented 3 years ago

Description Error occurs on first installation of connector with confluent-hub command during construction of docker connect image

Troubleshooting Validate every step in the troubleshooting section: https://docs.confluent.io/platform/current/tutorials/cp-demo/docs/index.html#troubleshooting -­­> Done Identify any existing issues that seem related: https://github.com/confluentinc/cp-demo/issues?q=is%3Aissue Nothing found Similar issue found in another repo - Unable to install kafka-connect-datagen:0.1.0

If applicable, please include the output of:

Environment

Probably caused by VPN software : ZScaler Client connector. Each outgoing request verified, if not whitelisted, will re-encrypt traffic with ZScacler certificates chain. Have to add URL explicitly. More output of common commands to help with troubleshooting

`curl -iv https://api.hub.confluent.io/api/plugins

< [{....}]) port 443 (#0)

< [{....}]`

`confluent-hub install cjmatta/kafka-connect-sse:1.0 --verbose --component-dir . --no-prompt Running in a verbose mode Running in a "--no-prompt" mode Implicit acceptance of the license below: The Apache License, Version 2.0 https://www.apache.org/licenses/LICENSE-2.0 Implicit confirmation of the question: You are about to install 'kafka-connect-sse' from Christopher Matta, as published on Confluent Hub. Downloading component Kafka Connect SSE 1.0, provided by Christopher Matta from Confluent Hub and installing into . Scanning path . Installation state for the kafka-connect-sse in the . is IMMEDIATE_DIRECTORY Implicit confirmation of the question: Do you want to uninstall existing version 1.0? Uninstalling kafka-connect-sse from . Deleting ./cjmatta-kafka-connect-sse Creating temporary directory Created temporary directory: /tmp/confluent-hub-tmp3005423504225809653 Copying data from input stream to file /tmp/confluent-hub-tmp3005423504225809653/cjmatta-kafka-connect-sse-1.0.zip 6375414 bytes saved to disk Creating directory recursively: . Unzipping file /tmp/confluent-hub-tmp3005423504225809653/cjmatta-kafka-connect-sse-1.0.zip to ./cjmatta-kafka-connect-sse Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/doc Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/etc Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/etc Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/etc Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/assets Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Creating directory recursively: ./cjmatta-kafka-connect-sse/lib Deleting /tmp/confluent-hub-tmp3005423504225809653 Client's installation type is: ARCHIVE Unable to detect Confluent Platform installation. Specify --component-dir and --worker-configs explicitly.

Error: Invalid options or arguments`

`openssl s_client -connect api.hub.confluent.io:443 -showcerts CONNECTED(00000003) depth=2 C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA verify return:1 depth=1 C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA verify return:1 depth=0 C = US, ST = CA, L = Mountain View, O = "Confluent, Inc.", OU = Information Technology, CN = *.confluent.io verify return:1

Certificate chain 0 s:C = US, ST = CA, L = Mountain View, O = "Confluent, Inc.", OU = Information Technology, CN = *.confluent.io i:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA -----BEGIN CERTIFICATE----- MIIG2TCCBcGgAwIBAgIQBSEG2pQlpLzV9DvuRT3amzANBgkqhkiG9w0BAQsFADBN MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMScwJQYDVQQDEx5E aWdpQ2VydCBTSEEyIFNlY3VyZSBTZXJ2ZXIgQ0EwHhcNMTkxMTE5MDAwMDAwWhcN MjIwMTA2MTIwMDAwWjCBhjELMAkGA1UEBhMCVVMxCzAJBgNVBAgTAkNBMRYwFAYD VQQHEw1Nb3VudGFpbiBWaWV3MRgwFgYDVQQKEw9Db25mbHVlbnQsIEluYy4xHzAd BgNVBAsTFkluZm9ybWF0aW9uIFRlY2hub2xvZ3kxFzAVBgNVBAMMDiouY29uZmx1 ZW50LmlvMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAv/CCPPyJVrUe uGFVkd5YOjha5mZXmBNr9XVeRLlZI6GSW3xenw3dEanrZTNLbWcznJI1i2NuevUz tfIJ6yY9BsRGwYZqRyjPnjtpEjsWKFOgeA8BUZ9SyHz46oV45HJJoXheDE0S8GIt 7Rli93O8XRwe2oevMfuLuLEUIul5nAFD3DklNEVf/8oe26hjJDE+9M4cBbMOCSZc /OIZOPWM9k7zSEUe0RGDvaY4VmZfI6rVR/j9NCZMbPHBD2QUomqsEux/eBKfn2oO W/PRsX1ZA7ZiL6ecmM4d3TV9E0zRyQJ2Ng3ruAdFKNSISYfL9WU6oWWlvzahOrpQ TNP0XS0qqQIDAQABo4IDeTCCA3UwHwYDVR0jBBgwFoAUD4BhHIIxYdUvKOeNRji0 LOHG2eIwHQYDVR0OBBYEFO6NGIuEXcnM1Bmg/WmJsw8wSjbPMD0GA1UdEQQ2MDSC DiouY29uZmx1ZW50Lmlvggxjb25mbHVlbnQuaW+CFGFwaS5odWIuY29uZmx1ZW50 LmlvMA4GA1UdDwEB/wQEAwIFoDAdBgNVHSUEFjAUBggrBgEFBQcDAQYIKwYBBQUH AwIwawYDVR0fBGQwYjAvoC2gK4YpaHR0cDovL2NybDMuZGlnaWNlcnQuY29tL3Nz Y2Etc2hhMi1nNi5jcmwwL6AtoCuGKWh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9z c2NhLXNoYTItZzYuY3JsMEwGA1UdIARFMEMwNwYJYIZIAYb9bAEBMCowKAYIKwYB BQUHAgEWHGh0dHBzOi8vd3d3LmRpZ2ljZXJ0LmNvbS9DUFMwCAYGZ4EMAQICMHwG CCsGAQUFBwEBBHAwbjAkBggrBgEFBQcwAYYYaHR0cDovL29jc3AuZGlnaWNlcnQu Y29tMEYGCCsGAQUFBzAChjpodHRwOi8vY2FjZXJ0cy5kaWdpY2VydC5jb20vRGln aUNlcnRTSEEyU2VjdXJlU2VydmVyQ0EuY3J0MAwGA1UdEwEB/wQCMAAwggF8Bgor BgEEAdZ5AgQCBIIBbASCAWgBZgB1AKS5CZC0GFgUh7sTosxncAo8NZgE+RvfuON3 zQ7IDdwQAAABboVvpCAAAAQDAEYwRAIgCcwDssIv93l6tCL2RrMXseuwYIkA9AgY +MYnVaRuoaICIFNTiZwRunQ4VDqBQu353jMtzir0oOtx5uv051NiueFyAHUAh3W/ 51l8+IxDmV+9827/Vo1HVjb/SrVgwbTq/16ggw8AAAFuhW+kaQAABAMARjBEAiBo jotH17OVshZfwNfBEuLgnJLKfPxou24Ve47vkF3R7gIgIWfgAu/m9seQAiOU9YZg ej4ZqNG0KRcxTivX5pi+8IAAdgBWFAaaL9fC7NP14b1Esj7HRna5vJkRXMDvlJhV 1onQ3QAAAW6Fb6RpAAAEAwBHMEUCIA2bUHo7qSwMLX5jm23AvXk/gWZkkLnfdBaX vR7HQ8c/AiEA/3Z2nNMiDMubFkR3+0hrbujQywf7iUESCqKchBJdJOIwDQYJKoZI hvcNAQELBQADggEBAHr3ANPL9GLNawc6wz2mZwVJ5XpPcUTrSbU83ikY9ZcLUBFp oaj+DyiZpiHCNt4u/sXB4Z8jM1ruYW3J/ADm15EY7HZhWQcdxA2ZL0sSdyHwqxbf biAfSd5/MaIAMy/sH5Oqtz/XxYVnpR52ZSA0x6RgcTa6DfRK95j3+ae7C6R3Ro2a bUppURcTp7vtrNIIhblDJfQDZKJlkM4LoUfRmdEdrs+yBgAOMPHEFxXwvQJn4xAR 3hYubfOSVU11bhZYGnL8g2sgPexzkregRakGsmAuaZ6VlvBTE9EVDkHD8EObnUO3 p/c0DhhTXJ9Xjs0U6VVYdi2H0eePYqTC1aDj5vU= -----END CERTIFICATE----- 1 s:C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA i:C = US, O = DigiCert Inc, OU = www.digicert.com, CN = DigiCert Global Root CA -----BEGIN CERTIFICATE----- MIIElDCCA3ygAwIBAgIQAf2j627KdciIQ4tyS8+8kTANBgkqhkiG9w0BAQsFADBh MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3 d3cuZGlnaWNlcnQuY29tMSAwHgYDVQQDExdEaWdpQ2VydCBHbG9iYWwgUm9vdCBD QTAeFw0xMzAzMDgxMjAwMDBaFw0yMzAzMDgxMjAwMDBaME0xCzAJBgNVBAYTAlVT MRUwEwYDVQQKEwxEaWdpQ2VydCBJbmMxJzAlBgNVBAMTHkRpZ2lDZXJ0IFNIQTIg U2VjdXJlIFNlcnZlciBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB ANyuWJBNwcQwFZA1W248ghX1LFy949v/cUP6ZCWA1O4Yok3wZtAKc24RmDYXZK83 nf36QYSvx6+M/hpzTc8zl5CilodTgyu5pnVILR1WN3vaMTIa16yrBvSqXUu3R0bd KpPDkC55gIDvEwRqFDu1m5K+wgdlTvza/P96rtxcflUxDOg5B6TXvi/TC2rSsd9f /ld0Uzs1gN2ujkSYs58O09rg1/RrKatEp0tYhG2SS4HD2nOLEpdIkARFdRrdNzGX kujNVA075ME/OV4uuPNcfhCOhkEAjUVmR7ChZc6gqikJTvOX6+guqw9ypzAO+sf0 /RR3w6RbKFfCs/mC/bdFWJsCAwEAAaOCAVowggFWMBIGA1UdEwEB/wQIMAYBAf8C AQAwDgYDVR0PAQH/BAQDAgGGMDQGCCsGAQUFBwEBBCgwJjAkBggrBgEFBQcwAYYY aHR0cDovL29jc3AuZGlnaWNlcnQuY29tMHsGA1UdHwR0MHIwN6A1oDOGMWh0dHA6 Ly9jcmwzLmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RDQS5jcmwwN6A1 oDOGMWh0dHA6Ly9jcmw0LmRpZ2ljZXJ0LmNvbS9EaWdpQ2VydEdsb2JhbFJvb3RD QS5jcmwwPQYDVR0gBDYwNDAyBgRVHSAAMCowKAYIKwYBBQUHAgEWHGh0dHBzOi8v d3d3LmRpZ2ljZXJ0LmNvbS9DUFMwHQYDVR0OBBYEFA+AYRyCMWHVLyjnjUY4tCzh xtniMB8GA1UdIwQYMBaAFAPeUDVW0Uy7ZvCj4hsbw5eyPdFVMA0GCSqGSIb3DQEB CwUAA4IBAQAjPt9L0jFCpbZ+QlwaRMxp0Wi0XUvgBCFsS+JtzLHgl4+mUwnNqipl 5TlPHoOlblyYoiQm5vuh7ZPHLgLGTUq/sELfeNqzqPlt/yGFUzZgTHbO7Djc1lGA 8MXW5dRNJ2Srm8c+cftIl7gzbckTB+6WohsYFfZcTEDts8Ls/3HB40f/1LkAtDdC 2iDJ6m6K7hQGrn2iWZiIqBtvLfTyyRRfJs8sjX7tN8Cp1Tm5gr8ZDOo0rwAhaPit c+LJMto4JQtV05od8GiG7S5BNO98pVAdvzr508EIDObtHopYJeS4d60tbvVS3bR0 j6tJLp07kzQoH3jOlOrHvdPJbRzeXDLz -----END CERTIFICATE-----

Server certificate subject=C = US, ST = CA, L = Mountain View, O = "Confluent, Inc.", OU = Information Technology, CN = *.confluent.io

issuer=C = US, O = DigiCert Inc, CN = DigiCert SHA2 Secure Server CA


No client certificate CA names sent Peer signing digest: SHA512 Peer signature type: RSA Server Temp Key: ECDH, P-256, 256 bits

SSL handshake has read 3443 bytes and written 438 bytes Verification: OK

New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256 Server public key is 2048 bit Secure Renegotiation IS supported Compression: NONE Expansion: NONE No ALPN negotiated SSL-Session: Protocol : TLSv1.2 Cipher : ECDHE-RSA-AES128-GCM-SHA256 Session-ID: B54041465020CE1A7660B18AA43C0691ACD073C37A504A74F82DA967CBD24FDC Session-ID-ctx: Master-Key: 81F206C6D5F1524DEB271391C9F71CCAC35A100390D716F320A47FF74E9D867F89CE2EA9E0E55BC631C747D7D032FEF3 PSK identity: None PSK identity hint: None SRP username: None Start Time: 1626108409 Timeout : 7200 (sec) Verify return code: 0 (ok) Extended master secret: no ---`

Any idea what could be possible be done to see which command , to which URL is breaking thing up ?

igal-ore-ssq commented 3 years ago

In the end arrived to resolve this issue by importing injected certificates into docker connect image, at Java key store, with Dockerfile COPY and RUN commands : USER root COPY ca-certificates/*.crt . RUN keytool -import -file zscaler_root_ca.crt -storepass changeit -alias zscale_root_ca -noprompt -cacerts RUN keytool -import -file '"zscaler_intermediate_rootca(zscalerthreenet)(t)_".crt' -storepass changeit -alias zscale_inter_root_ca -noprompt -cacerts RUN keytool -import -file 'zscaler_intermediate_rootca(zscalerthree_net)emailaddress=_support@zscaler_com.crt' -storepass changeit -alias zscale_interim_root_catemp -noprompt -cacerts USER appuser

and some other OpenSSL commands to extract certificates from traffic.