Closed jborean93 closed 3 years ago
I'm 99% positive this happened on certificate renewal: Issued On Monday, March 22, 2021 at 8:18:08 PM. And I recall something similar happening the last year.
FWIW if the client is forced not to use SNI, the certificate received is one that belongs to Fastly. So this assumption may be incorrect (since the error pip gives mentions the whole bunch of SANs):
$ openssl s_client -connect files.pythonhosted.org:443 -noservername
CONNECTED(00000003)
depth=1 C = BE, O = GlobalSign nv-sa, CN = GlobalSign CloudSSL CA - SHA256 - G3
verify return:1
depth=0 C = US, ST = California, L = San Francisco, O = "Fastly, Inc", CN = r.ssl.fastly.net
verify return:1
---
Certificate chain
0 s:C = US, ST = California, L = San Francisco, O = "Fastly, Inc", CN = r.ssl.fastly.net
i:C = BE, O = GlobalSign nv-sa, CN = GlobalSign CloudSSL CA - SHA256 - G3
1 s:C = BE, O = GlobalSign nv-sa, CN = GlobalSign CloudSSL CA - SHA256 - G3
i:C = BE, O = GlobalSign nv-sa, OU = Root CA, CN = GlobalSign Root CA
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIIRjCCBy6gAwIBAgIMJrEVtnw2k+M+uodaMA0GCSqGSIb3DQEBCwUAMFcxCzAJ
BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMS0wKwYDVQQDEyRH
bG9iYWxTaWduIENsb3VkU1NMIENBIC0gU0hBMjU2IC0gRzMwHhcNMjEwMzIzMTYy
NzU5WhcNMjEwNDI4MTkyMDI1WjBrMQswCQYDVQQGEwJVUzETMBEGA1UECAwKQ2Fs
aWZvcm5pYTEWMBQGA1UEBwwNU2FuIEZyYW5jaXNjbzEUMBIGA1UECgwLRmFzdGx5
LCBJbmMxGTAXBgNVBAMMEHIuc3NsLmZhc3RseS5uZXQwggEiMA0GCSqGSIb3DQEB
AQUAA4IBDwAwggEKAoIBAQDTciJwVUeYpeFv0YzWACjrrAg7Qy2yWqLNrGXTJu0m
0bWfeB9kA3gY3gny66LF6W/hYVXtf+oCneDWIpm7eO8DogP+k9H2uwOkCnEXOGog
Jj1QfWnlqYQSpK0HRZ8idm6EglJANXBqK/G+Nbnqb0P68MsdaTKLo1zHyRcEncbZ
OnWAgYkZ1s5CyNQmNhn741JrkXuEcLjuRi5oFP252w8qhcxLAzI9T7pfym9ERcvM
kwZUXUIv9z0qrU3HikASfL6TC2O1qfHwVd4RvR74WOdDHkS02AQWx3NpVbfBLI53
GM/f59vocj3z6cV5TmUcbgrvYt2Lm215mjIOefyzxqlxAgMBAAGjggT8MIIE+DAO
BgNVHQ8BAf8EBAMCBaAwgYoGCCsGAQUFBwEBBH4wfDBCBggrBgEFBQcwAoY2aHR0
cDovL3NlY3VyZS5nbG9iYWxzaWduLmNvbS9jYWNlcnQvY2xvdWRzc2xzaGEyZzMu
Y3J0MDYGCCsGAQUFBzABhipodHRwOi8vb2NzcDIuZ2xvYmFsc2lnbi5jb20vY2xv
dWRzc2xzaGEyZzMwVgYDVR0gBE8wTTBBBgkrBgEEAaAyARQwNDAyBggrBgEFBQcC
ARYmaHR0cHM6Ly93d3cuZ2xvYmFsc2lnbi5jb20vcmVwb3NpdG9yeS8wCAYGZ4EM
AQICMAkGA1UdEwQCMAAwggITBgNVHREEggIKMIICBoIQci5zc2wuZmFzdGx5Lm5l
dIIQKi5jYXRjaHBvaW50LmNvbYIIKi5jbm4uaW+CFSouZG9sbGFyc2hhdmVjbHVi
LmNvbYILKi5lYXRlci5jb22CFiouZmFzdGx5LnBpY21vbmtleS5jb22CHCouZmls
ZXMuc2F5bWVkaWEtY29udGVudC5jb22CCCouZnQuY29tghIqLm1lZXR1cHN0YXRp
Yy5jb22CCSoubmZsLmNvbYIKKi5wYWdhci5tZYIPKi5waWNtb25rZXkuY29tgg4q
LnJlYWxzZWxmLmNvbYIOKi5zYm5hdGlvbi5jb22CCyouc2hha3IuY29tghAqLnN0
cmVhbWFibGUuY29tggwqLnN1cmZseS5jb22CDioudGhldmVyZ2UuY29tgg8qLnRo
cmlsbGlzdC5jb22CDSoudm94LWNkbi5jb22CCSoudm94LmNvbYIOKi52b3htZWRp
YS5jb22CCWVhdGVyLmNvbYIGZnQuY29tgghpLmdzZS5pb4INcGljbW9ua2V5LmNv
bYIMcmVhbHNlbGYuY29tghRzdGF0aWMud2l4c3RhdGljLmNvbYIOc3RyZWFtYWJs
ZS5jb22CCnN1cmZseS5jb22CDHRoZXZlcmdlLmNvbYILdm94LWNkbi5jb22CB3Zv
eC5jb22CDnd3dy5qb3llbnQuY29tMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEF
BQcDAjAfBgNVHSMEGDAWgBSpK4fhziRHOxu/z4U3AlWdDZRY5jAdBgNVHQ4EFgQU
8WPkxcyYMOyDfYryy2qnZTH95r8wggF+BgorBgEEAdZ5AgQCBIIBbgSCAWoBaAB3
AG9Tdqwx8DEZ2JkApFEV/3cVHBHZAsEAKQaNsgiaN9kTAAABeF/qGDcAAAQDAEgw
RgIhAISIL5bX4HXeCVTN9RbOxoZIlTvPAHqVtLOXvaWaB5CoAiEAqHMB0QyTJcCE
URiJHrCHHMnZqwQKcjYdWTCJtCGxDhcAdQBVgdTCFpA2AUrqC5tXPFPwwOQ4eHAl
CBcvo6odBxPTDAAAAXhf6hhbAAAEAwBGMEQCIBS3R/dFRh/9j1FsCop6mMpJrKzR
6ysCD7Xu8y9Tvw/LAiBWt+ZMtB2Gmn45V4g0dOHOfibHmMGBi8zk97qQ31/YTgB2
APZclC/RdzAiFFQYCDCUVo7jTRMZM7/fDC8gC8xO8WTjAAABeF/qGD0AAAQDAEcw
RQIhAJPkbvvRT7iQ84A9dO0D9GmPUnKl4bU5Dc7LR2Yl92NPAiBQAXFavccJiwDK
wBHGyYFREW5LtP/KsgerPlkXjVSdRzANBgkqhkiG9w0BAQsFAAOCAQEAPKadZIFK
/u6SZ1XsHpkTh1NNBx0cr8Xb8xsH/lnJlaGI1mWAVvlE1hhJd2r3q02ioekpFGuO
G2UbMfTwhP+SQyrPZKhGpx7qIl3sMkkN21KwiIO2HViFc9636bvIDh4ji5h1vCFF
pBXtVT0Lkf2m09XpGr5f74EgMh5rraajBjQSHF9QMdv4CpIJGhVmG9Wj2Ax7jDLN
69wxjjZdhpevf83mzIpeCZyhuiW9hobmenC9NctQ9iUDgR+EXBhtowlCnVJpr8fI
fnmpEtU1cB6e24VUYHKm8wEOhNZFvY7CkC5k7PLqp+MUn0l5RAQZmYLT+retrOLI
JFjSFOTBhoiETw==
-----END CERTIFICATE-----
subject=C = US, ST = California, L = San Francisco, O = "Fastly, Inc", CN = r.ssl.fastly.net
issuer=C = BE, O = GlobalSign nv-sa, CN = GlobalSign CloudSSL CA - SHA256 - G3
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 3917 bytes and written 386 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES128-GCM-SHA256
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES128-GCM-SHA256
Session-ID: 8FA1BEC376BE18016E656D713FC4A2C39672ADB80F586BAA91449D2A6F9F84CF
Session-ID-ctx:
Master-Key: 2E2E4F67911254DA30DBC96A3AAC51CF2B91577F6B321C8F162A572B9DCF72DE5528C066669872ACC581B1A2B3D251F2
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket lifetime hint: 7200 (seconds)
TLS session ticket:
0000 - 40 f8 5a 5a 24 15 b1 fb-4d 9e 1e c3 93 59 46 a4 @.ZZ$...M....YF.
0010 - da c4 eb 79 f4 ba 80 66-25 72 ae 26 38 f0 3e 63 ...y...f%r.&8.>c
0020 - e7 b1 16 35 4b f5 22 9c-6e ff 80 99 26 5f 8d a8 ...5K.".n...&_..
0030 - 98 75 08 b4 6b 79 1f 8b-b5 6e 92 cb da c8 c4 68 .u..ky...n.....h
0040 - fd e9 48 57 58 c8 db 73-3d 19 9f 39 58 98 61 90 ..HWX..s=..9X.a.
0050 - d4 3c 6c 6d e9 08 91 ac-5f 50 6b 91 a0 02 67 54 .<lm...._Pk...gT
0060 - b1 d3 a0 49 03 2c 6e 25-28 4a c0 4e 11 e6 40 65 ...I.,n%(J.N..@e
0070 - 17 34 98 e3 e5 91 ba 66-a5 78 bf ec f6 9b af c7 .4.....f.x......
0080 - be f5 9d 90 6a 18 16 a5-4d e7 7e 32 60 3c e1 27 ....j...M.~2`<.'
0090 - e0 d3 1d 29 c0 2e 3a 82-63 31 03 87 15 84 26 e6 ...)..:.c1....&.
Start Time: 1616536553
Timeout : 7200 (sec)
Verify return code: 0 (ok)
Extended master secret: yes
---
@ewdurbin could you take a look at this, please?
Testing files.pythonhosted.org with Qualys SSL Labs confirms SNI is required and the alternative names do not match.
The latest test results for one of the servers (151.101.1.63) reports multiple relevant issues:
I've emailed Fastly support about this issue, in case it is a problem on their end, since nothing shows up on their status page.
This is the response I received from Fastly support:
Thanks for chiming in and excellent question. After reviewing the SSL Labs results, you're seeing the mismatch in the certificate because that is the fallback certificate for non-SNI requests. As of right now, it appears you're on a shared certificate, and to remedy this issue, you will need to look into having a dedicated IP space, where you have control of how traffic is served for non-SNI requests.
Unfortunately this doesn't explain why this just started failing today.
Fastly support sent a second response:
Can you tell me if the requests you are sending have the SNI servername header being sent? We did our migration this week where the shared cert is no longer the only cert on the IP address you've cnamed to. If that is the case, then you will get that servername mismatch.
And then another:
This issue should be resolved for the time being.
Please note well that the SAN cert that your SAN is on will be deprecated by Globalsign by May 2021. At that point, if your issue is with sending the SNI servername header, you may need to contact your Sales Rep or Account Manager here at Fastly for the options to accommodate non-SNI requests.
I've confirmed that pip install
is working again now -- although based on the above statements, only temporarily.
This is indeed #978, we were made aware that they planned to deprecate SNI support but missed further timeline information, which was delivered in a separate email thread, or we would have made announcements sooner.
Consolidating into #978.
I wrote a get-pip
for Python 2.6 (GNU/Linux x86_64) that should workaround this problem by forcing pip
to use pyOpenSSL
:
https://gist.github.com/molinav/3b4f623edc5793154a0bdd9a78e739e9
It uses a similar approach to get-pip
attaching all the required packages inside the script itself. After execution, pip
will be able to reach PyPI under Python 2.6. I cannot reference it in #978 too because the comments are closed there.
For whoever that may be still interested on this, I moved my previous gist to an actual repository in GitHub: https://github.com/pylegacy/get-pip-pyopenssl
And it is possible to obtain get-pip-pyopenssl
by just doing (under GNU/Linux):
wget http://pylegacy.org/hub/get-pip-pyopenssl.py
python get-pip-pyopenssl.py
Current supported versions include Python 2.6 and 2.7 under GNU/Linux and Windows, but it should be easily extendable in the future to more Python versions.
My Platform
Python 2.6 using
pip==9.0.3
.The issue here is that
pip install ...
doesn't support SNI due to the ancient version of Python that is in use (2.6'sssl
module doesn't support SNI). Totally understand that SNI support on 2.6 requirespyOpenSSL
and the injection code to be run but the issue is the SNI requirement seemed to have just popped out of nowhere. This was working just fine but now is not so I was wondering if this was an intentional change?Fastly Debug
DNS Resolution
Traceroutes / IPv4
Traceroutes / IPv6 (If available)
N/A
HTTPS Requests / IPv4
HTTPS Requests / IPv6 (If available)
N/A
TLS Debug / IPv4
TLS Debug / IPv4 without SNI
Note: the cert shown by pip is presented here because SNI is disabled in the openssl call to replicate Python 2.6's behaviour
TLS Debug / IPv6 (If available)
N/A
Code of Conduct