TLS 1.2 problem with Microsoft server

[nikomi]

Hi,

recently Microsoft seems to have changed something on it's WNS server(s). Clients using your TLS library fail to connect when TLS 1.2 is implicitly or explicitly on the list of supported versions. This can be reproduced using your tls-debug tool:

The following two commands show the failure to connect: tls-simpleclient -v -d db3.notify.windows.com 443 tls-simpleclient -v -d db3.notify.windows.com 443 --tls12

The following two commands on the other hand work fine: tls-simpleclient -v -d db3.notify.windows.com 443 --tls11 tls-simpleclient -v -d db3.notify.windows.com 443 --tls10

On other Microsoft servers - e.g. login.live.com - this problem does not surface.

[meteficha]

I'm having exactly the same issues with the host at api.sandbox.mangopay.com.

Ping @JPMoresmau @jcristovao @alanpog.

[meteficha]

More details for anyone googling this:

# Fails
$ tls-simpleclient -v -d api.sandbox.mangopay.com 443 --tls12
sending query:
GET / HTTP/1.0

debug: >> Handshake [ClientHello TLS12 (ClientRandom "U\241\SYN\155I\NAK\240J\164\180\193\128\215\SYN2\247\\\199\175c\148\v\147\145\128\DC3\155ae!\205\v") (Session Nothing) [107,103,57,51,56,50,47,53,4,5,10] [0] [(0,"\NUL\ESC\NUL\NUL\CANapi.sandbox.mangopay.com"),(65281,"\NUL")] Nothing]
debug: >> Alert [(AlertLevel_Fatal,InternalError)]
tls-simpleclient: send: resource vanished (Broken pipe)

# Works
$ tls-simpleclient -v -d api.sandbox.mangopay.com 443 --tls11
sending query:
GET / HTTP/1.0

debug: >> Handshake [ClientHello TLS11 (ClientRandom "\160AX\151\181(\SYN,r\STX\188^yA,\254\230\161o\197\190\253\160o\DEL#:\195%\225k\234") (Session Nothing) [107,103,57,51,56,50,47,53,4,5,10] [0] [(0,"\NUL\ESC\NUL\NUL\CANapi.sandbox.mangopay.com"),(65281,"\NUL")] Nothing]
debug: << Handshake [ServerHello TLS11 (ServerRandom "Tp\182\190T\214b4O\177]xZ\238\DC2N\242\DELv\166YVJ\145\147\b\224+Z>\204\SO") (Session (Just "a&\NUL\NUL\188V|\164\f\159\138r9M\194\233\159\211.\220\203\203e\209e\140\134\172']K ")) 53 0 [(65281,"\NUL"),(0,"")],Certificates [...]

For some reason the server doesn't like our handshake?

[meteficha]

FWIW, using @kazu-yamamoto's WIP support for ECDHE and AEAD (https://github.com/kazu-yamamoto/hs-tls/commit/d1c9978, https://github.com/kazu-yamamoto/hs-tls/commit/4f14cd3, https://github.com/kazu-yamamoto/hs-crypto-pubkey/commit/8ad949aa971ed6265278f587dde51eb97aea4251) does not change these outcomes.

[kazu-yamamoto]

FYI. My ECDHE implementation is not completed. Currently, the server side only. I will implement the client side in the next week.

[vincenthz]

ok, I found the problem with this. after a long 'match-what-openssl-is-doing' debugging session (which sadly I didn't manage to get the right intuition, so I ended up implementing a bunch of things, until the last unmatching thing)

so, the result, is that the extension number 13 (Signature algorithm) seems necessary for a bunch of servers to actually bother replying at all. As far as I'm concerned that seems again, a violation of the spec, but heh, not the first one with TLS.

I'll commit a fix soon, including some others stuff necessary for EC* implementations (EcPointFormat and EllipticCurve extensions)

[meteficha]

Thanks for your work, @vincenthz, that's awesome :).

[vincenthz]

ok, master should be working now @nikomi, @meteficha.

[vincenthz]

@kazu-yamamoto I should have looked at your tree first, looks like I've reimplemented both TLS EC extensions. Your EC extensions are probably better as they integrate with the crypto repository, whereas mine were just a copy paste of the spec.

[kazu-yamamoto]

@vincenthz No problem. My EC implementation is not completed. So, I did not send a pull request yet. I will catch up your new code.

Before that, it would be highly appreciated if you give comments on https://github.com/vincenthz/hs-tls/pull/90.

[nikomi]

Thanx for your work @vinzenthz!

Unluckily it seems the current master tls-v1.2.13-14-g334394a seems not to solve the problem:

$ tls-simpleclient -v -d db3.notify.windows.com 443 sending query: GET / HTTP/1.0

debug: >> Handshake [ClientHello TLS12 (ClientRandom "\199\177\ENQ\168]\202\204|\SO\254\252\225\151j\163\235&\196\153\156\235\190\170\216\213\251\154\163@\252\226\r") (Session Nothing) [107,103,57,51,56,50,47,53,4,5,10] [0] [(0,"\NUL\EM\NUL\NUL\SYNdb3.notify.windows.com"),(65281,"\NUL")] Nothing] debug: >> Alert [(AlertLevel_Fatal,InternalError)] tls-simpleclient: send: resource vanished (Broken pipe)

[vincenthz]

@nikomi: you're not running tls-simpleclient with the latest tls; the newly added extension doesn't appear in the log above. Very likely you forgot to install tls, or forgot to re-install tls-debug.

[nikomi]

Mea culpa, I have sandbox builds and had not realized that tls-debug did not use the new tls-core. Now I sandbox add-sourced the new tls-core and it works like a charm :-)

Thanx for all your effort and support!

When do you expect to release this to Hackage?

[vincenthz]

I want to review and add @kazu-yamamoto's support for AEAD and then I'll make a point release. this is either going to be tomorrow or the day after.

[johntyree]

@vincenthz, it seems like the support for (some of?) Microsoft's servers has broken again, but only recently, I think. I definitely had a working example last weekend and now it's back to broken pipe.

@nikomi's example works fine above:

$ ./.cabal-sandbox/bin/tls-simpleclient -v db3.notify.windows.com                                                                       
sending query:
GET / HTTP/1.0

HTTP/1.1 405 Method Not Allowed
Keep-Alive: true
Content-Length: 0
X-WNS-MSG-ID: 5E6EC73C361163B0
X-WNS-DEBUG-TRACE: DB3WNS2011036
Strict-Transport-Security: max-age=31536000; includeSubDomains
Date: Fri, 12 Dec 2014 01:47:51 GMT
Connection: close

but the azure datamarket no longer does:

$ .cabal-sandbox/bin/tls-simpleclient -v -d datamarket.accesscontrol.windows.net 443
sending query:
GET / HTTP/1.0

debug: >> Handshake [ClientHello TLS12 (ClientRandom "\DC3\151\160O\167\162S0\243\179\219\f\135\183u\192Z0_X\157\194\247\143\170Y`bv\183q\237") (Session Nothing) [107,103,57,51,56,50,47,53,4,5,10] [0] [(0,"\NUL'\NUL\NUL$datamarket.accesscontrol.windows.net"),(65281,"\NUL"),(13,"\NUL\n\ACK\SOH\ENQ\SOH\EOT\SOH\ETX\SOH\STX\STX")] Nothing]
debug: >> Alert [(AlertLevel_Fatal,InternalError)]
tls-simpleclient: send: resource vanished (Broken pipe)

[johntyree]

I don't know if it's helpful for protocol snooping, but I have a tcpdump -w packet file for both curl successfully handshaking with that server and tls-simpleclient failing to do so.

They are pretty small but I haven't studied TLS so I didn't make much of them when I took a look myself.

http://john.bitsurge.net/public/pkts/

[nikomi]

Thanx for the new versions!

The Windows server I originally reported works fine now, as does api.sandbox.mangopay.com (reported by meteficha).

On the downside the Windows OAuth server login.live.com - which worked before - has stopped working using TLS package 1.2.16:

dist/dist-sandbox-65a52b7c/build/tls-simpleclient/tls-simpleclient -dv login.live.com 443
sending query:
GET / HTTP/1.0

debug: >> Handshake [ClientHello TLS12 (ClientRandom "\188F\248n\241.\234\ro\NAKZ\220\&6\n\221\219\132\&9#\ACKz\FS\248\202\221\220\RS\190\193\193\228\169") (Session Nothing) [107,103,57,51,56,50,47,53,4,5,10] [0] [(0,"\NUL\DC1\NUL\NUL\SOlogin.live.com"),(65281,"\NUL"),(13,"\NUL\n\ACK\SOH\ENQ\SOH\EOT\SOH\ETX\SOH\STX\STX")] Nothing]
debug: >> Alert [(AlertLevel_Fatal,InternalError)]
tls-simpleclient: send: resource vanished (Broken pipe)

TLS protocol version 1.1 still works agains login.live.com.

Could you please take a look at this? thanx in advance, Nik

[RandalliLama]

I looked in to this a bit and found the Windows dev who owns SChannel, the windows TLS implementation. This is what he had to say:

Your test client sends a TLS1.2 ClientHello with SignatureAlgorithms extension that has the following algorithm pairs:
00 0a – don’t know what this is; must be garbage bytes? Schannel ignores this pair.
06 01 – SHA512, RSA
05 01 - SHA384, RSA
04 01 – SHA256, RSA
03 01 – SHA 224, RSA – this one is unusual, we don’t support (and therefore ignore) it.
02 02 – SHA1, DSA – this one is not very useful in practice since no one uses DSA certs.

Once schannel filters out non-existent 00 0a and unsupported 03 01, we’re left with:
06 01 – SHA512, RSA
05 01 - SHA384, RSA
04 01 – SHA256, RSA
02 02 – SHA1, DSA

TLS 1.2 RFC says:
“If the client provided a "signature_algorithms" extension, then all
   certificates provided by the server MUST be signed by a
   hash/signature algorithm pair that appears in that extension.”

Now if the TLS server only has a SHA1, RSA certificate, the connection with your client will fail. A server with SHA2,RSA or SHA1, DSA cert will allow this client to connect (assuming everything else works). E.g. your test client seems to work with Bing, which presents SHA2, RSA cert.

I’d recommend that Haskell figure out the 00 0a issue, and send 02 01 instead of (or in addition to) 02 02. SHA1 is weak, but it does not make much sense to allow SHA1 with DSA, but not with RSA.

[vincenthz]

ok this is easy enough to fix. Although I'm not sure about this "garbage bytes"; this is not garbage, this is the length of the variable vectors that follows it. i.e.:

Variable-length vectors are defined by specifying a subrange of legal
   lengths, inclusively, using the notation <floor..ceiling>.  When
   these are encoded, the actual length precedes the vector's contents
   in the byte stream.

....

SignatureAndHashAlgorithm
    supported_signature_algorithms<2..2^16-1>;

[vincenthz]

Thanks @RandalliLama for the investigation. previous commands that failed seems to work now.

I've released tls-1.2.17 with the one liner fix (adding RSA/SHA1)

[RandalliLama]

@vincenthz That was quick! I agree on the garbage bytes comment. I think that was a misunderstanding of some sort.

[vincenthz]

great, that make sense. I assumed as such, as I've done this in the past when looking at raw bytes one too many times.

[nikomi]

Thanx for all the effort, guys!

[johntyree]

:tada: hooray!