search

chrismarget / certbot-asa

Cisco ASA plugin for certbot
Other
23 stars 5 forks source link

tls: handshake failure #1

Open pana67 opened 7 years ago

pana67 commented 7 years ago

Hi, I am trying to make work enrolment lets encrypt certificate for our ASA firewall / VPN gateway. I am able to communicate with ASA REST API, I installed I hope all required software on internal linux server (Debian),but when I run the command for enrolment I obtain error:

Thank you for any help.

chrismarget commented 7 years ago

I've seen this error myself recently.

The issue was that when LE's authenticator connected to the ASA to validate the challenge certificate, it wasn't able to establish a TLS session with the ASA.

I didn't figure out what was wrong, but suspect it's an SSL cipher support issue on the ASA. At the time I had the problem, the same LE authenticator (same IP address, anyway) was able to talk fine with one of my ASAs, but not with the other.

There was no time to debug the problem at that time, so I didn't pursue it further.

What version of software is running on the ASA? I'll see if I can replicate the problem.

pana67 commented 7 years ago

Hi Chris,

I am running the last available ASA-OS - 9.8(1) ( I need at least 9.7.1 as there is VTI interface configured) I am quite skilled in ASA so I can try to debug SSL connection on the ASA side – that does not apply for my skills in linux :-o

Regards

Pavel

From: Chris Marget [mailto:notifications@github.com] Sent: Friday, June 30, 2017 5:14 PM To: chrismarget/certbot-asa certbot-asa@noreply.github.com Cc: Navrátil Pavel navratil@networksys.cz; Author author@noreply.github.com Subject: Re: [chrismarget/certbot-asa] tls: handshake failure (#1)

I've seen this error myself recently.

The issue was that when LE's authenticator connected to the ASA to validate the challenge certificate, it wasn't able to establish a TLS session with the ASA.

I didn't figure out what was wrong, but suspect it's an SSL cipher support issue on the ASA. At the time I had the problem, the same LE authenticator (same IP address, anyway) was able to talk fine with one of my ASAs, but not with the other.

There was no time to debug the problem at that time, so I didn't pursue it further.

What version of software is running on the ASA? I'll see if I can replicate the problem.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/chrismarget/certbot-asa/issues/1#issuecomment-312294636, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcauW_okzholqS3NVL5wINDH3DQd5BxMks5sJRDEgaJpZM4OKqpe.

chrismarget commented 7 years ago

So, if the failure is happening where I think it's happening...

If you run 'show run | inc trustpoint|trust-point' repeatedly on the ASA while the plugin is operating, you should see a temporary trustpoint and SNI configuration appear briefly.

Once those are installed, we're just waiting for the LE authenticator to come look at them.

At this point, you should be able to see the TLS failure if you're watching packets (fairly easy to spot), or with the right debug incantation on the ASA (I'm not sure what that might be).

I'll fire up an ASA in the lab.

pana67 commented 7 years ago

Hi,

you are right – I can see the temporary acme challenge trustpoint for a while.

fw# sh runn | in acme crypto ca trustpoint acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 keypair acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 crypto ca certificate chain acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 ssl trust-point acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 domain 85bd3442b8ece3d01183ae375610bea5.6b7ac57b435eac0a38c306f42d8b61bd.acme.invalid

I tried to set ssl debug – I can see error

error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher@s3_srvr.c:2053

in time the server should check the certificate

I hope this can help

Regards

Pavel

From: Chris Marget [mailto:notifications@github.com] Sent: Friday, June 30, 2017 5:54 PM To: chrismarget/certbot-asa certbot-asa@noreply.github.com Cc: Navrátil Pavel navratil@networksys.cz; Author author@noreply.github.com Subject: Re: [chrismarget/certbot-asa] tls: handshake failure (#1)

So, if the failure is happening where I think it's happening...

If you run 'show run | inc trustpoint|trust-point' repeatedly on the ASA while the plugin is operating, you should see a temporary trustpoint and SNI configuration appear briefly.

Once those are installed, we're just waiting for the LE authenticator to come look at them.

At this point, you should be able to see the TLS failure if you're watching packets (fairly easy to spot), or with the right debug incantation on the ASA (I'm not sure what that might be).

I'll fire up an ASA in the lab.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/chrismarget/certbot-asa/issues/1#issuecomment-312304968, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcauW7BRLKGi7dShxWedc9-KN_0JzoDfks5sJRorgaJpZM4OKqpe.

pana67 commented 7 years ago

Hi,

I found it will probably be problem in cypher set - the error message "no shared cipher" should mean there is no supported encryption cipher on ASA - I tried to allow all kinds of cypher on ASA but it ends with the same error.

Regards

Pavel

From: Navrátil Pavel Sent: Friday, June 30, 2017 7:21 PM To: 'chrismarget/certbot-asa' reply@reply.github.com; chrismarget/certbot-asa certbot-asa@noreply.github.com Cc: Author author@noreply.github.com Subject: RE: [chrismarget/certbot-asa] tls: handshake failure (#1)

Hi,

you are right – I can see the temporary acme challenge trustpoint for a while.

fw# sh runn | in acme crypto ca trustpoint acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 keypair acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 crypto ca certificate chain acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 ssl trust-point acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 domain 85bd3442b8ece3d01183ae375610bea5.6b7ac57b435eac0a38c306f42d8b61bd.acme.invalid

I tried to set ssl debug – I can see error

error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher@s3_srvr.c:2053mailto:cipher@s3_srvr.c:2053

in time the server should check the certificate

I hope this can help

Regards

Pavel

From: Chris Marget [mailto:notifications@github.com] Sent: Friday, June 30, 2017 5:54 PM To: chrismarget/certbot-asa certbot-asa@noreply.github.com<mailto:certbot-asa@noreply.github.com> Cc: Navrátil Pavel navratil@networksys.cz<mailto:navratil@networksys.cz>; Author author@noreply.github.com<mailto:author@noreply.github.com> Subject: Re: [chrismarget/certbot-asa] tls: handshake failure (#1)

So, if the failure is happening where I think it's happening...

If you run 'show run | inc trustpoint|trust-point' repeatedly on the ASA while the plugin is operating, you should see a temporary trustpoint and SNI configuration appear briefly.

Once those are installed, we're just waiting for the LE authenticator to come look at them.

At this point, you should be able to see the TLS failure if you're watching packets (fairly easy to spot), or with the right debug incantation on the ASA (I'm not sure what that might be).

I'll fire up an ASA in the lab.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/chrismarget/certbot-asa/issues/1#issuecomment-312304968, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcauW7BRLKGi7dShxWedc9-KN_0JzoDfks5sJRorgaJpZM4OKqpe.

pana67 commented 7 years ago

And one more probably useful info - these are the set supported by default on ASA:

fw# sh ssl ciphers Current cipher configuration: default (medium): ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-GCM-SHA384 AES256-GCM-SHA384 ECDHE-ECDSA-AES256-SHA384 ECDHE-RSA-AES256-SHA384 DHE-RSA-AES256-SHA256 AES256-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-GCM-SHA256 AES128-GCM-SHA256 ECDHE-ECDSA-AES128-SHA256 ECDHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA256 AES128-SHA256 DHE-RSA-AES256-SHA AES256-SHA DHE-RSA-AES128-SHA AES128-SHA tlsv1 (medium): DHE-RSA-AES256-SHA AES256-SHA DHE-RSA-AES128-SHA AES128-SHA tlsv1.1 (medium): DHE-RSA-AES256-SHA AES256-SHA DHE-RSA-AES128-SHA AES128-SHA tlsv1.2 (medium): ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 DHE-RSA-AES256-GCM-SHA384 AES256-GCM-SHA384 ECDHE-ECDSA-AES256-SHA384 ECDHE-RSA-AES256-SHA384 DHE-RSA-AES256-SHA256 AES256-SHA256 ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES128-GCM-SHA256 AES128-GCM-SHA256 ECDHE-ECDSA-AES128-SHA256 ECDHE-RSA-AES128-SHA256 DHE-RSA-AES128-SHA256 AES128-SHA256 DHE-RSA-AES256-SHA AES256-SHA DHE-RSA-AES128-SHA AES128-SHA dtlsv1 (medium): DHE-RSA-AES256-SHA AES256-SHA DHE-RSA-AES128-SHA AES128-SHA

Regards

Pavel

From: Navrátil Pavel Sent: Friday, June 30, 2017 7:36 PM To: 'chrismarget/certbot-asa' reply@reply.github.com; 'chrismarget/certbot-asa' certbot-asa@noreply.github.com Cc: 'Author' author@noreply.github.com Subject: RE: [chrismarget/certbot-asa] tls: handshake failure (#1)

Hi,

I found it will probably be problem in cypher set - the error message "no shared cipher" should mean there is no supported encryption cipher on ASA - I tried to allow all kinds of cypher on ASA but it ends with the same error.

Regards

Pavel

From: Navrátil Pavel Sent: Friday, June 30, 2017 7:21 PM To: 'chrismarget/certbot-asa' reply@reply.github.com; chrismarget/certbot-asa certbot-asa@noreply.github.com<mailto:certbot-asa@noreply.github.com> Cc: Author author@noreply.github.com<mailto:author@noreply.github.com> Subject: RE: [chrismarget/certbot-asa] tls: handshake failure (#1)

Hi,

you are right – I can see the temporary acme challenge trustpoint for a while.

fw# sh runn | in acme crypto ca trustpoint acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 keypair acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 crypto ca certificate chain acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 ssl trust-point acme_challenge_0ce0e19e6b60fd64350fbc9277aad043 domain 85bd3442b8ece3d01183ae375610bea5.6b7ac57b435eac0a38c306f42d8b61bd.acme.invalid

I tried to set ssl debug – I can see error

error:1408A0C1:SSL routines:ssl3_get_client_hello:no shared cipher@s3_srvr.c:2053mailto:cipher@s3_srvr.c:2053

in time the server should check the certificate

I hope this can help

Regards

Pavel

From: Chris Marget [mailto:notifications@github.com] Sent: Friday, June 30, 2017 5:54 PM To: chrismarget/certbot-asa certbot-asa@noreply.github.com<mailto:certbot-asa@noreply.github.com> Cc: Navrátil Pavel navratil@networksys.cz<mailto:navratil@networksys.cz>; Author author@noreply.github.com<mailto:author@noreply.github.com> Subject: Re: [chrismarget/certbot-asa] tls: handshake failure (#1)

So, if the failure is happening where I think it's happening...

If you run 'show run | inc trustpoint|trust-point' repeatedly on the ASA while the plugin is operating, you should see a temporary trustpoint and SNI configuration appear briefly.

Once those are installed, we're just waiting for the LE authenticator to come look at them.

At this point, you should be able to see the TLS failure if you're watching packets (fairly easy to spot), or with the right debug incantation on the ASA (I'm not sure what that might be).

I'll fire up an ASA in the lab.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/chrismarget/certbot-asa/issues/1#issuecomment-312304968, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcauW7BRLKGi7dShxWedc9-KN_0JzoDfks5sJRorgaJpZM4OKqpe.

chrismarget commented 7 years ago

Yup, this is where I wound up last time it happened.

It's not really an issue with the plugin, but the TLS service on the ASA rejecting LE's TLS client.

I couldn't figure it out last time, but I intend to take another crack at it.

chrismarget commented 7 years ago

Pavel, can you test something for me?

You'll have to be quick: During the interval when the ASA is configured to serve up the challenge certificate, but before LE gets around to checking... Please try to delete the ssl trust-point acme-challenge_blah domain blah line from the ASA configuration?

My hope is that the error will change from something about TLS handshaking to a challenge validation failure (indicating that TLS succeeded).

I'm finding that with that directive in place I have trouble with the TLS handshake on my test box.

chrismarget commented 7 years ago

I think we're running into this Cisco bug. An ASA upgrade is going to be required.

pana67 commented 7 years ago

Hi Chris,

I just post a request for help on Cisco Support Forum - will see if somebody answer to it. https://supportforums.cisco.com/discussion/13325616/ssl-handshake-failure

I sniffed the SSL communication - enclosing the file with it - you can check it weather you will be able to get some useful info from it.

Regards

Pavel

From: Chris Marget [mailto:notifications@github.com] Sent: Saturday, July 1, 2017 12:57 PM To: chrismarget/certbot-asa certbot-asa@noreply.github.com Cc: Navrátil Pavel navratil@networksys.cz; Author author@noreply.github.com Subject: Re: [chrismarget/certbot-asa] tls: handshake failure (#1)

I think we're running into thishttps://bst.cloudapps.cisco.com/bugsearch/bug/CSCve20346 Cisco bug. An ASA upgrade is going to be required.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/chrismarget/certbot-asa/issues/1#issuecomment-312425434, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AcauW6uOFzLAM4Pe-b-SvIY7gxjPX28oks5sJiX3gaJpZM4OKqpe.

chrismarget commented 7 years ago

I'm confident that the bug I mentioned is your problem. There's a couple of options for verifying that:

  1. Downgrade to 9.3(2). The problem should go away.

  2. Manually:

The s_client command should fail with SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure. That's the bug.

Remove the ssl trust-point line, or change the -servername argument to some other value and the s_client command should succeed (meaning you don't get your Linux shell prompt back). At that point, type GET / HTTP/1.0 <CR><CR> and the ASA should spit some HTML at you (through the TLS session created by openssl).

It seems that any TLS connection which matches the SNI filter specified by the ssl trust-point <blah> domain <blah> directive will have a problem because of this bug. Frankly, I'm surprised you got past the openssl s_client step in the plugin's readme, because it relies on SNI.