search

openwrt / packages

Community maintained packages for OpenWrt. Documentation for submitting pull requests is in CONTRIBUTING.md
GNU General Public License v2.0
4.05k stars 3.5k forks source link

Curl and Wget fail on some https sites, openssl does not #16652

Closed Jeronimo17 closed 3 years ago

Jeronimo17 commented 3 years ago

Maintainer: @stangri @tripolar Environment: MediaTek MT7621 ver:1 eco:3, Newifi-D2, OpenWrt 21.02.0 r16279-5cc0535800

Description:

root@OpenWrt:~# curl https://www.boe.es -v
* SSL_connect failed with error -313: received alert fatal error
curl: (35) SSL_connect failed with error -313: received alert fatal error

root@OpenWrt:/etc/ssl# curl --cacert /etc/ssl/certs/ca-certificates.crt https://www.boe.es -v
* SSL_connect failed with error -313: received alert fatal error
curl: (35) SSL_connect failed with error -313: received alert fatal error
root@OpenWrt:/etc/ssl# curl -k https://www.boe.es -v
* SSL_connect failed with error -313: received alert fatal error
curl: (35) SSL_connect failed with error -313: received alert fatal error

root@OpenWrt:~#  wget https://www.boe.es
Downloading 'https://www.boe.es'
Connecting to 81.89.32.200:443
Connection error: Connection failed

root@OpenWrt:~# openssl s_client -connect boe.es:443
CONNECTED(00000003)
depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
verify return:1
depth=1 C = NL, O = GEANT Vereniging, CN = GEANT OV RSA CA 4
verify return:1
depth=0 C = ES, ST = Madrid, L = Madrid, O = Bolet\C3\ADn Oficial del Estado, CN = www.boe.es
verify return:1
---
Certificate chain
 0 s:C = ES, ST = Madrid, L = Madrid, O = Bolet\C3\ADn Oficial del Estado, CN = www.boe.es
   i:C = NL, O = GEANT Vereniging, CN = GEANT OV RSA CA 4
 1 s:C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
   i:C = GB, ST = Greater Manchester, L = Salford, O = Comodo CA Limited, CN = AAA Certificate Services
 2 s:C = NL, O = GEANT Vereniging, CN = GEANT OV RSA CA 4
   i:C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST Network, CN = USERTrust RSA Certification Authority
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIHXTCCBUWgAwIBAgIQPEtQ2N70Uh2V7I2bY6yULzANBgkqhkiG9w0BAQwFADBE
MQswCQYDVQQGEwJOTDEZMBcGA1UEChMQR0VBTlQgVmVyZW5pZ2luZzEaMBgGA1UE
AxMRR0VBTlQgT1YgUlNBIENBIDQwHhcNMjEwNjA0MDAwMDAwWhcNMjIwNjA0MjM1
OTU5WjBqMQswCQYDVQQGEwJFUzEPMA0GA1UECBMGTWFkcmlkMQ8wDQYDVQQHEwZN
YWRyaWQxJDAiBgNVBAoMG0JvbGV0w61uIE9maWNpYWwgZGVsIEVzdGFkbzETMBEG
A1UEAxMKd3d3LmJvZS5lczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AJx5XC3jV6x2EXpamd2DT0Ve2hgpIFY9EICg0bprRkKaQlY/wOcnDodhxtu9G1qp
F92n++oaA1CdHj0rwD1wvfo5YHbioqYzU+Xli47nR7MuMCboS1oyw4DkRJu41kF5
hNjSS3dtwFIJxM4M5rh23jPovSoluOMPPMdRDT/PfKwF3++zRey6bnwXspW1k1cw
ZFDihp/baQ2g4l9Bb0n9wvsJmoFWmGcXfeOhiRBApPVWc1a4sY4iBkxQwtkVygfR
9Byv0cJ38/oiwKPZfnwSaQ6m8TlIyeTFLpbukwzawK+/dEY0KI/KdhFOywiXkWWb
GpFDVPusHsbxLIeNcP4tOD8CAwEAAaOCAyMwggMfMB8GA1UdIwQYMBaAFG8dNUkQ
bDL6WaCevIroH5W+cXoMMB0GA1UdDgQWBBR/V48Za8M3y9Eh0QkviktFAlKwjzAO
BgNVHQ8BAf8EBAMCBaAwDAYDVR0TAQH/BAIwADAdBgNVHSUEFjAUBggrBgEFBQcD
AQYIKwYBBQUHAwIwSQYDVR0gBEIwQDA0BgsrBgEEAbIxAQICTzAlMCMGCCsGAQUF
BwIBFhdodHRwczovL3NlY3RpZ28uY29tL0NQUzAIBgZngQwBAgIwPwYDVR0fBDgw
NjA0oDKgMIYuaHR0cDovL0dFQU5ULmNybC5zZWN0aWdvLmNvbS9HRUFOVE9WUlNB
Q0E0LmNybDB1BggrBgEFBQcBAQRpMGcwOgYIKwYBBQUHMAKGLmh0dHA6Ly9HRUFO
VC5jcnQuc2VjdGlnby5jb20vR0VBTlRPVlJTQUNBNC5jcnQwKQYIKwYBBQUHMAGG
HWh0dHA6Ly9HRUFOVC5vY3NwLnNlY3RpZ28uY29tMB0GA1UdEQQWMBSCCnd3dy5i
b2UuZXOCBmJvZS5lczCCAXwGCisGAQQB1nkCBAIEggFsBIIBaAFmAHUARqVV63X6
kSAwtaKJafTzfREsQXS+/Um4havy/HD+bUcAAAF51hzz8wAABAMARjBEAiAuGYex
YfuLpmSt95WqwHhD+WPKCeXhq9O24XHA8WrykAIgGzzkGki9XRlDaCZESct0RH4d
x/ghWpLqE8ZH2yQcgJgAdQBByMqx3yJGShDGoToJQodeTjGLGwPr60vHaPCQYpYG
9gAAAXnWHPO1AAAEAwBGMEQCIEmjQQkakdyxNCTxzlTNwA57mdVTcRYlMFxG+8Cl
lqPJAiBAe7UMti3gUd3NRN6HBZQF6/g8uE8NZaF0WQlOgDWHRAB2ACl5vvCeOTkh
8FZzn2Old+W+V32cYAr4+U1dJlwlXceEAAABedYc844AAAQDAEcwRQIhALlQTiRV
6KufONaUWeZ2+Cwi7yb9RdToKNlYGU0mVcFSAiA9Jop0ATSxzYqL5ezvw5ruRHiQ
3Wzml0vFsc+JUvp1czANBgkqhkiG9w0BAQwFAAOCAgEAVEYq8vvXL0agO0Q1D57f
GakslpI73x3at3qKJauf6eUmH9TULb9PPB/DzjsvRI7NfsJ92sRsPX94w8HIZarf
+4md0fssu3vgbWDHeBVc/qcCLFpps2ciHlDwYCBW6eTUOhFiCGreyom/8PKg6HaE
YNWTtDIRiQCxE/HvL65CiZYLvLRFhoxRPuVt+8zpXVr3krIrzadVMquTnh7eyB2X
AJq+v3i7ROkfeijd+JVp2PtZn5bXaDQCj9sdArBqvBJ7ZAYjHqzxrrRHxwt7oRBE
M3eF9J0wVyTv4UBbBtW2wgR5Va+es5uDLOSyFSB672FGzLO2Lamnz2ycTFP/nq2h
HvVSSRRXxJow7AcOTIL7jFDa5f/EMw0flirDM2Dpm1OcHNeZqmztK+2V6zUAX7tl
ywi8gERkczrRrbR3l05Aq5Eq5TJlCH06ouBiYvQervVkvZjW9rh5fJzE3XFNNNw6
2UD0jNcCIZOPmYau450uxYv9mRpm8lvkW31KOY3yso8YR9W9K7++aFIQXBa+5xvz
z0wwCoqFyJKOZIGAdk0K41V31LXWuhRKnMonuAdV4tb84zn2lHIUpOkkX0wNnj/t
89seCsbYP8CtnraTYaJ3DZxZbaN2sb4LiDe6K63u4RE/u/72sAgVQmxS/G/DPztv
EYU8wwbvyR7FfVlMRjC7UnY=
-----END CERTIFICATE-----
subject=C = ES, ST = Madrid, L = Madrid, O = Bolet\C3\ADn Oficial del Estado, CN = www.boe.es

issuer=C = NL, O = GEANT Vereniging, CN = GEANT OV RSA CA 4

---
No client certificate CA names sent
---
SSL handshake has read 5409 bytes and written 626 bytes
Verification: OK
---
New, TLSv1.2, Cipher is AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES256-GCM-SHA384
    Session-ID: 5598071FDD1206F5861C0B2152985E7D5CE66FC56DFEC22F46E91A557C446B28
    Session-ID-ctx: 
    Master-Key: 1B4C5BDF8BD9DF9B95D12C18ED73DFC4CA10C2076E8188DCD5A98EA70B16FE67D78B87EE5F7AE172E41F00D113761490
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 73 1f 53 cc 0a 98 17 9a-d7 a6 37 a5 06 85 48 45   s.S.......7...HE
    0010 - 25 20 4b 6b 8d 7d 97 ae-b2 5a ce 4d 65 99 6f 45   % Kk.}...Z.Me.oE
    0020 - c2 fb 31 fe 24 b9 2b b6-00 57 14 d0 19 cf 9e d7   ..1.$.+..W......
    0030 - bf 98 15 39 1b 59 c8 4f-53 96 e3 a1 57 cf c3 51   ...9.Y.OS...W..Q
    0040 - db 7c 05 7a 91 4e 29 98-4e aa 1c 07 b5 2b a5 84   .|.z.N).N....+..
    0050 - 56 b6 c5 c4 46 82 11 ee-81 29 9e b3 16 c9 a6 86   V...F....)......
    0060 - 80 e1 27 dd f9 42 a3 3f-af 80 1c 1c 5f 4a 54 46   ..'..B.?...._JTF
    0070 - 92 af dd 68 2c dc 18 11-31 4c 92 78 0e 1c 2f a6   ...h,...1L.x../.
    0080 - c2 b2 ed eb 62 95 02 a9-53 b0 76 05 62 b4 d5 05   ....b...S.v.b...
    0090 - dc 8a 63 d6 bb f0 df 35-9c 92 b6 e5 6f 04 40 53   ..c....5....o.@S
    00a0 - c2 e6 66 1d b4 da 33 20-4f de 82 3a 97 92 d8 5d   ..f...3 O..:...]

    Start Time: 1631890194
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: yes
---
GET / HTTP/1.0
Host:boe.es

HTTP/1.1 200 OK
Date: Fri, 17 Sep 2021 14:48:41 GMT
Server: Apache
x-frame-options: SAMEORIGIN
Content-Security-Policy: default-src 'none'; script-src 'unsafe-inline' 'unsafe-eval' 'self' ; connect-src 'self' ; img-src 'self' eur-lex.europa.eu data: ; style-src 'unsafe-inline' 'self' *.boe.es ; font-src 'self' ; child-src 'self'  www.youtube.com afirma: ; object-src 'self' ; media-src 'self'
Content-Type: text/html; charset=UTF-8
X-Varnish: 542296087 542868340
Age: 103
Via: 1.1 varnish-v4
Content-Length: 22112
Accept-Ranges: bytes
Connection: close

I think with 19.07 it didn't fail Thank you

stangri commented 3 years ago

Thank you for creating this. By default in OpenWrt curl uses WolfSSL, however when compiled to rely on OpenSSL, it connects to the web-site in question just fine:

# curl https://www.boe.es -v
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
> GET / HTTP/1.1
> Host: www.boe.es
> User-Agent: curl/7.79.0
> Accept: */*

@neheb @champtar this is not the first time when there were issues with curl's depdendency on WolfSSL, I believe in some cases it causes memory leaks when using https-dns-proxy when the openssl-based version works fine. What would be the proper media to discuss switching to OpenSSL instead?

neheb commented 3 years ago

The original reason for switching to wolfssl was because that's what wpad was switched to in order to support WPA3.

Personally, I'd move it back to the older default of mbedTLS.

This is also not the first time there were issues with wolfSSL. Transmission for example does not work with it.

micmac1 commented 3 years ago

Checking boe.es with "nmap --script ssl-enum-ciphers -p 443 www.boe.es" shows

TLSv1.0 TLSv1.1 TLSv1.2

For all three nmap emits a warning: "Forward Secrecy not supported by any cipher"

wolfSSL's README says that it no longer supports static key cipher suites by default. I think that forward secrecy means not using a static suite, at least that's what I gather from a few web searches. So to summarize if a host doesn't support forward secrecy then wolfSSL will not be able to connect to it, unless it's compiled with

WOLFSSL_STATIC_DH WOLFSSL_STATIC_RSA WOLFSSL_STATIC_PSK

which OpenWrt doesn't do (and wolfSSL upstream says it's deprecated).

Is this correct?

Jeronimo17 commented 3 years ago

Sorry for putting this here, a temporary trick for me has been to install the 19.07 version ipk: curl_7.66.0-3_mipsel_24kc.ipk libcurl4_7.66.0-3_mipsel_24kc.ipk libnghttp2-14_1.41.0-1_mipsel_24kc.ipk

micmac1 commented 3 years ago

Checking boe.es with "nmap --script ssl-enum-ciphers -p 443 www.boe.es" shows

TLSv1.0 TLSv1.1 TLSv1.2

For all three nmap emits a warning: "Forward Secrecy not supported by any cipher"

wolfSSL's README says that it no longer supports static key cipher suites by default. I think that forward secrecy means not using a static suite, at least that's what I gather from a few web searches. So to summarize if a host doesn't support forward secrecy then wolfSSL will not be able to connect to it, unless it's compiled with

WOLFSSL_STATIC_DH WOLFSSL_STATIC_RSA WOLFSSL_STATIC_PSK

which OpenWrt doesn't do (and wolfSSL upstream says it's deprecated).

Is this correct?

wolfSSL README says regarding static key cipher suites: "They also lower your security by removing PFS." So yes, with wolfSSL you can only connect to PFS-enabled sites be default (the site in question is not PFS-enabled).

With default wolfSSL build:

su -s /bin/sh -c "curl https://www.boe.es -v" nobody
* SSL_connect failed with error -313: received alert fatal error
curl: (35) SSL_connect failed with error -313: received alert fatal error

With changed build:

su -s /bin/sh -c "curl https://www.boe.es -v" nobody
> GET / HTTP/1.1
> Host: www.boe.es
> User-Agent: curl/7.78.0
> Accept: */*
> 
< HTTP/1.1 200 OK

The only change I made in wolfSSL Makefile was adding "-DWOLFSSL_STATIC_RSA" to TARGET_CFLAGS. I compiled it in an 21.02 SDK.

Making this change in wolfSSL is an alternative to what @stangri is proposing in issue #16674. The size difference in libwolfssl is only 1 KByte:

-rw-r--r-- 1 xy xy 420K 21. Sep 12:59 bin/packages/mips_24kc/base/libwolfssl4.7.0.66253b90_4.7.0-stable-2_mips_24kc.ipk

-rw-r--r-- 1 xy xy 421K 21. Sep 21:39 bin/packages/mips_24kc/base/libwolfssl4.7.0.66253b90_4.7.0-stable-3_mips_24kc.ipk

micmac1 commented 3 years ago

I should have written "is maybe an alternative to what @stangri is proposing". While this small change makes this issue go away, I don't know if it has any positive effect on the https-dns-proxy mem leak or the problem with Transmission.

stangri commented 3 years ago

wolfSSL README says regarding static key cipher suites: "They also lower your security by removing PFS." So yes, with wolfSSL you can only connect to PFS-enabled sites be default (the site in question is not PFS-enabled).

I've tried to look this up and couldn't find relevant information: is there a way to estimate the percentage of the sites which only support static key ciphers?

Also, regardless of the other issues with WolfSSL, maybe it's not a worst idea to send a PR to re-enable static key ciphers support in OpenWrt version of WolfSSL in the mean time. @micmac1 would you be willing to do that?

micmac1 commented 3 years ago

wolfSSL README says regarding static key cipher suites: "They also lower your security by removing PFS." So yes, with wolfSSL you can only connect to PFS-enabled sites be default (the site in question is not PFS-enabled).

I've tried to look this up and couldn't find relevant information: is there a way to estimate the percentage of the sites which only support static key ciphers?

There are probably some surveys available online.

Also, regardless of the other issues with WolfSSL, maybe it's not a worst idea to send a PR to re-enable static key ciphers support in OpenWrt version of WolfSSL in the mean time. Would you be willing to do that?

No, don't see myself getting into this. I have no opinion on this. I was just curious.

Jeronimo17 commented 3 years ago

OpenWrt 21.02.1 Still the same, same trick use curl from 19.07.1

Jeronimo17 commented 2 years ago

OpenWrt 21.02.2 Still the same, same trick use curl from 19.07.1

stangri commented 2 years ago

Tagging maintainer: @cotequeiroz.

I believe there were multiple changes necessary (on top of updating wolfssl) for a range of issues reported about same time as this, I'm not sure if the support for alt chains and static rsa has been enabled in master?

Jeronimo17 commented 2 years ago

OpenWrt 21.02.3 Still the same, same trick use curl from 19.07.1

goldriver2 commented 1 year ago

curl https://www.boe.es -v

Hi @Jeronimo17, can you please detail how you are installing curl from 19.07.01 ?

I understand you need thoses 3 files: curl_7.66.0-3_mipsel_24kc.ipk libcurl4_7.66.0-3_mipsel_24kc.ipk libnghttp2-14_1.41.0-1_mipsel_24kc.ipk

But have no clue how you installed them, I tried opkg install ... but it does not work

Jeronimo17 commented 1 year ago

It already works and I don't do it but previously I think I deleted curl and libcurl.

opkg uninstall ...