search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
984 stars 76 forks source link

Can't establish SSL connection #143

Open kvaps opened 4 years ago

kvaps commented 4 years ago

Hi we just faced with non-working csi-plugin:

I0510 16:18:10.596227       1 connection.go:183] GRPC request: {"node_id":"m8c1","volume_id":"pvc-f34d05ad-e947-4d04-8fcb-6dea48b0863a"}
I0510 16:18:10.610557       1 connection.go:185] GRPC response: {}
I0510 16:18:10.611857       1 connection.go:186] GRPC error: rpc error: code = Internal desc = ControllerUnpublishVolume failed for pvc-ce743221-38a0-496b-8037-36a01897e1a2: Get "https://linstor-controller:3371/v1/resource-definitions/pvc-ce743221-38a0-496b-8037-36a01897e1a2": EOF
I0510 16:18:10.611907       1 csi_handler.go:578] Saving detach error to "csi-28431d1daf8e959fc0415c5f1e983bedd9b629255659b6a81dd31306ddf82938"
I0510 16:18:10.614159       1 connection.go:185] GRPC response: {}
I0510 16:18:10.615285       1 connection.go:186] GRPC error: rpc error: code = Internal desc = ControllerUnpublishVolume failed for pvc-f34d05ad-e947-4d04-8fcb-6dea48b0863a: Get "https://linstor-controller:3371/v1/resource-definitions/pvc-f34d05ad-e947-4d04-8fcb-6dea48b0863a": EOF
I0510 16:18:10.615315       1 csi_handler.go:578] Saving detach error to "csi-3de5ddabe7642757f58a7a0fa5385e49d79e9f615a9fb0317d0dfd011d154ba0"
I0510 16:18:10.618112       1 csi_handler.go:589] Saved detach error to "csi-28431d1daf8e959fc0415c5f1e983bedd9b629255659b6a81dd31306ddf82938"
I0510 16:18:10.618164       1 csi_handler.go:222] Error processing "csi-28431d1daf8e959fc0415c5f1e983bedd9b629255659b6a81dd31306ddf82938": failed to detach: rpc error: code = Internal desc = ControllerUnpublishVolume failed for pvc-ce743221-38a0-496b-8037-36a01897e1a2: Get "https://linstor-controller:3371/v1/resource-definitions/pvc-ce743221-38a0-496b-8037-36a01897e1a2": EOF
I0510 16:18:10.623898       1 csi_handler.go:589] Saved detach error to "csi-3de5ddabe7642757f58a7a0fa5385e49d79e9f615a9fb0317d0dfd011d154ba0"
I0510 16:18:10.623942       1 csi_handler.go:222] Error processing "csi-3de5ddabe7642757f58a7a0fa5385e49d79e9f615a9fb0317d0dfd011d154ba0": failed to detach: rpc error: code = Internal desc = ControllerUnpublishVolume failed for pvc-f34d05ad-e947-4d04-8fcb-6dea48b0863a: Get "https://linstor-controller:3371/v1/resource-definitions/pvc-f34d05ad-e947-4d04-8fcb-6dea48b0863a": EOF

The weird thing is that curl also can't perform the requests:

# curl --version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets 
# curl -k --cacert /tls/ca.crt --cert /tls/tls.crt --key /tls/tls.key https://localhost:3371/v1/controller/version
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.
# curl -k --cacert /tls/ca.crt --cert /tls/tls.crt --key /tls/tls.key https://localhost:3371/v1/controller/version
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.
# curl -k --cacert /tls/ca.crt --cert /tls/tls.crt --key /tls/tls.key https://localhost:3371/v1/controller/version
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.
# curl -k --cacert /tls/ca.crt --cert /tls/tls.crt --key /tls/tls.key https://localhost:3371/v1/controller/version
curl: (35) gnutls_handshake() failed: Error in the pull function.
# curl -k --cacert /tls/ca.crt --cert /tls/tls.crt --key /tls/tls.key https://localhost:3371/v1/controller/version
curl: (35) gnutls_handshake() failed: Error in the pull function.
# curl -k --cacert /tls/ca.crt --cert /tls/tls.crt --key /tls/tls.key https://localhost:3371/v1/controller/version
curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.

However linstor client and curl on another machine and working without any problems:

# curl --version
curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL 
# curl --cacert /etc/linstor/ca.crt --cert /etc/linstor/tls.crt --key /etc/linstor/tls.key https://linstor-controller.linstor:3371/v1/controller/version                                           
{"version":"1.7.0","git_hash":"106dec17e9e4e34e292bf537ff01274b14ffddb7","build_time":"2020-05-07T22:55:44+00:00","rest_api_version":"1.1.0"}
# echo -e "GET /v1/controller/version HTTP/1.1\r\nHost: example.com\r\n\r\n" | openssl s_client -quiet -CAfile /tls/ca.crt -cert /tls/tls.crt -key /tls/tls.key -connect 127.0.0.1:3371
depth=1 CN = linstor-ca
verify return:1
depth=0 CN = linstor-controller
verify return:1
HTTP/1.1 200 OK
Access-Control-Allow-Origin: *
Access-Control-Allow-Headers: origin, content-type, accept, authorization
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS, HEAD
Content-Type: application/json
Content-Length: 141

{"version":"1.7.0","git_hash":"106dec17e9e4e34e292bf537ff01274b14ffddb7","build_time":"2020-05-07T22:55:44+00:00","rest_api_version":"1.1.0"}

If I try to debug connection using openssl s_client it is also working:

# echo -e "GET /v1/controller/version HTTP/1.1\r\nHost: example.com\r\n\r\n" | openssl s_client -CAfile /tls/ca.crt -cert /tls/tls.crt -key /tls/tls.key -connect 127.0.0.1:3371
CONNECTED(00000003)
depth=1 CN = linstor-ca
verify return:1
depth=0 CN = linstor-controller
verify return:1
---
Certificate chain
 0 s:/CN=linstor-controller
   i:/CN=linstor-ca
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIDajCCAlKgAwIBAgIRALVS1YisLlA1DtFcehHZWDAwDQYJKoZIhvcNAQELBQAw
FTETMBEGA1UEAxMKbGluc3Rvci1jYTAeFw0yMDA1MTAwMDE2NDBaFw0zMDA1MDgw
MDE2NDBaMB0xGzAZBgNVBAMTEmxpbnN0b3ItY29udHJvbGxlcjCCASIwDQYJKoZI
hvcNAQEBBQADggEPADCCAQoCggEBAM7CxlHNyvH8eh05zggMzdBgWj9KLa7HeEtt
OHQIcZOPcXVgfMe6gJ25cDjqo4exYfzalkspjb1oUbcgEe5tnPggB2YDpDHMUg5x
PRZHis+kjQ3JtGbmNmNrJ+I9oYptv2oyxDOMWNT3tGHZ+lEJNinCGbf9E4wPlJeo
+TcwZtrf67jaalPLbm5RwX+hBDM1444D2wXCDJKO+YahxVhMluzPgZN1zuj/gja3
yq/4UWXHiVOFXtHuh5OfX38l3Fd0u+D1dYPj4mSqfa4oA0rTHlvv1qmzxpvnFpBE
jiLihXjHl5sTPP01OwiLkN1fSdWNC+qu6bpouN3yuWfOF/d6ppUCAwEAAaOBrDCB
qTAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMC
MAwGA1UdEwEB/wQCMAAwagYDVR0RBGMwYYISbGluc3Rvci1jb250cm9sbGVyghps
aW5zdG9yLWNvbnRyb2xsZXIubGluc3RvcoIebGluc3Rvci1jb250cm9sbGVyLmxp
bnN0b3Iuc3Zjgglsb2NhbGhvc3SHBH8AAAEwDQYJKoZIhvcNAQELBQADggEBAEyp
aHdEu6+Qrhp6M+1fxeN5WRVJEVWwErVJ32Rk8iTmiiIccVbPsv6qVpXK4OROPygf
unfIT5Umm3fCDOAU09QF8TUu6IO19LBS5FxIlg3SLvpDS4gIZBIAcyHUeuKPfIc/
/U3hT0vx5EHK4kNm6k0Nw7cmaDy6TjwQlZGxQDJqNCAoAw6YFUzPpBO6auZOlmDZ
grwOHFKDt7x5Gm6wjdzQfeYmWXi5KOn7ojChXI2w2BEZEE7yLS29pnRAfxwNmfM/
hOdaUDUnDIcbvdhImuF0VqFNgCTcLzr9H0byinVMUVJ8Hwmo66hb8sBwaxNH8ulN
0PvJe77dm2B7o4SCmMA=
-----END CERTIFICATE-----
subject=/CN=linstor-controller
issuer=/CN=linstor-ca
---
Acceptable client certificate CA names
/CN=linstor-controller
Client Certificate Types: RSA sign, DSA sign, ECDSA sign
Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1
Shared Requested Signature Algorithms: ECDSA+SHA512:RSA+SHA512:ECDSA+SHA384:RSA+SHA384:ECDSA+SHA256:RSA+SHA256:DSA+SHA256:ECDSA+SHA224:RSA+SHA224:DSA+SHA224:ECDSA+SHA1:RSA+SHA1:DSA+SHA1
Peer signing digest: SHA512
Server Temp Key: ECDH, P-256, 256 bits
---
SSL handshake has read 1433 bytes and written 2249 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : ECDHE-RSA-AES256-GCM-SHA384
    Session-ID: 5EB828A0BDA4FFC25CEDB898B67E048C9E5F8E31DC9ED885CDCB1B6533480E49
    Session-ID-ctx: 
    Master-Key: D5204FECD63E8E2EAEBE2A9E330F79C3C8CC1AC7F0EC17680634839B57D2A952A7242FAA1974B9FD99A84AA3F4A1BC1E
    Key-Arg   : None
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    Start Time: 1589127328
    Timeout   : 300 (sec)
    Verify return code: 0 (ok)
---
DONE
kvaps commented 4 years ago

Problem with curl is persist on ubuntu xenial, but on debian stretch is working fine. However linstor-csi still showing EOF even on debian stretch