search

pact-foundation / pact_broker-client

A Ruby and CLI client for the Pact Broker. Publish and retrieve pacts and verification results.
MIT License
69 stars 47 forks source link

SSL certification issue for `can-i-deploy` #63

Closed sowmyav24 closed 4 years ago

sowmyav24 commented 4 years ago

We have been trying to use https://hub.docker.com/r/pactfoundation/pact-cli to use it's can-i-deploy mechanism. It keeps throwing Error making request - OpenSSL::SSL::SSLError SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) /usr/lib/ruby/gems/2.5.0/gems/pact_broker-client-1.20.0/lib/pact_broker/client/matrix.rb:13:inget', attempt 1 of 3`

We even tried passing SSL cert as an environment parameter but no luck.

Could someone help us as to how we can move forward ?

mefellows commented 4 years ago

Assuming your broker is using a self signed certificate? What environment variables have you tried so far?

sowmyav24 commented 4 years ago

The pact-broker is using an org signed certificate. I tried setting both SSL_CERT_FILE and SSL_CERT_DIR to the certificate

sowmyav24 commented 4 years ago

This was the command used docker run --rm \ -e PACT_BROKER_BASE_URL \ -e SSL_CERT_FILE \ pactfoundation/pact-cli:latest \ broker can-i-deploy \ --pacticipant ParticipantName \ --latest

bethesque commented 4 years ago

You'd need to mount the cert file into the docker image using -v and then give it the path to the cert file inside the docker container, not the one on the host file system.

bethesque commented 4 years ago

I've added docs here: https://github.com/pact-foundation/pact-ruby-cli#using-a-custom-certificate

sowmyav24 commented 4 years ago

Sure, I'll try that. But in addition to using docker, I tried the standalone cli

./pact-broker can-i-deploy --pacticipant NAME --broker-base-url $PACT_BROKER_BASE_URL --version 0.0.0

And had exported SSL_CERT_FILE as env variable. This also threw similar errror

sowmyav24 commented 4 years ago

The issue persists for docker image after following the documentation

sowmyav24 commented 4 years ago

The SSL certificate seems to be ok. I tried checking using openssl verify <name>.pem

bethesque commented 4 years ago

Can you double check your SSL cert file by running a curl command against your own broker using your own cert file like this?

SSL_CERT_FILE=/your/cert.pem curl -v https://your-broker:443

sowmyav24 commented 4 years ago

Hi @bethesque Yes it passed

bethesque commented 4 years ago

I'm out of ideas! All my tests with a self signed cert work locally. Is your pact broker exposed on the public internet?

sowmyav24 commented 4 years ago

Nope, the pact-broker is not exposed publicly.

bethesque commented 4 years ago

@sowmyav24 I'm trying to get a sharable local set up happening here: https://github.com/DiUS/pact_broker-docker/commit/e58f59ab362ccf25a06f453bb7cff8e29bfc1d1d

Unfortunately, I'm getting a 502 Bad Gateway from ngnix.

@mefellows I'm trying to get the ngnix SSL configuration working on the original docker compose file. I know it was already broken when we forked the dius/pact-broker-docker project to pact-foundation/pact-broker-docker, because I remember trying it out at that stage. Do you have any ideas?

mefellows commented 4 years ago

Bugger. I'll take a look tomorrow Beth.

On Tue, Jan 28, 2020, 9:42 PM Beth Skurrie notifications@github.com wrote:

@sowmyav24 https://github.com/sowmyav24 I'm trying to get a sharable local set up happening here: DiUS/pact_broker-docker@e58f59a https://github.com/DiUS/pact_broker-docker/commit/e58f59ab362ccf25a06f453bb7cff8e29bfc1d1d

Unfortunately, I'm getting a 502 Bad Gateway from ngnix.

@mefellows https://github.com/mefellows I'm trying to get the ngnix SSL configuration working on the original docker compose file. I know it was already broken when we forked the dius/pact-broker-docker project to pact-foundation/pact-broker-docker, because I remember trying it out at that stage. Do you have any ideas?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pact-foundation/pact_broker-client/issues/63?email_source=notifications&email_token=AAANFDCKRMYCD2KOHGXAIUTRAAD23A5CNFSM4KL3MXL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKC2TGA#issuecomment-579185048, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANFDHFP6X4EGKPAADIKNTRAAD23ANCNFSM4KL3MXLQ .

bethesque commented 4 years ago

Ok, I've got this working! Damn, certificates are a pain! @sowmyav24 consider using https://letsencrypt.org/ to get a free non-self-signed certificate - you might be able to skip all this faffing around.

Please checkout this branch @sowmyav24 https://github.com/DiUS/pact_broker-docker/tree/spike/ssl

Then run docker-compose up postgres pact_broker_app pact_broker in one window. Once everything has started up, run docker-compose up can_i_deploy. EDIT - ignore this, see my next comment

I get this output:

Attaching to dius-pact-broker-docker_can_i_deploy_1
can_i_deploy_1     | Computer says yes \o/
can_i_deploy_1     |
can_i_deploy_1     | CONSUMER    | C.VERSION                      | PROVIDER    | P.VERSION                      | SUCCESS?
can_i_deploy_1     | ------------|--------------------------------|-------------|--------------------------------|---------
can_i_deploy_1     | Example App | 5556b8149bf8bac76bc30f50a8a... | Example API | 4fdf20082263d4c5038355a3b73... | true
can_i_deploy_1     |
can_i_deploy_1     | All required verification results are published and successful
dius-pact-broker-docker_can_i_deploy_1 exited with code 0

If I comment out the SSL_CERT_FILE line then I get:

can_i_deploy_1     | Error making request - OpenSSL::SSL::SSLError SSL_connect returned=1 errno=0 state=error: certificate verify failed (unable to get local issuer certificate) /usr/lib/ruby/gems/2.5.0/gems/pact_broker-client-1.18.0/lib/pact_broker/client/matrix.rb:13:in `get', attempt 1 of 3
bethesque commented 4 years ago

Update: I've set the dependencies between the docker containers so you should just be able to run docker-compose up now, rather than starting them separately.

sowmyav24 commented 4 years ago

@bethesque Sure i'll have a look at this. Thanks !

sowmyav24 commented 4 years ago

The above works, But the pact broker set up here has a CA signed certificate and I am not sure if i can use a self signed certificate in this case. While i keep debugging why in my case the ssl certificate does not work, is there an alternate way to disable SSL from pact-cli similar to pact-broker's config of disable_ssl_verification

bethesque commented 4 years ago

The CA certificate is self signed though. I don't think it should make a difference, but I could be wrong. Let me see if I can generate a self signed cert to test it. Either way, we've proven that the environment variable is being correctly passed through to the Ruby code from the outside world.

There is a dirty dirty hack you can use to disable SSL verification across the platform in Ruby, but I'd have to release a code change. Let me think about how I could do it.

sowmyav24 commented 4 years ago

There is a dirty dirty hack you can use to disable SSL verification across the platform in Ruby, but I'd have to release a code change. Let me think about how I could do it.

Agreed ! This can be the last last resort.

I'll also re-verify the certs from my end.

mefellows commented 4 years ago

You might have luck by concatenating the ca public certificate (and any intermediates) as well as the actual certificate into a single file.

I'm not sure of the internals of the Ruby setup, but often it's the CA that needs trusting self signed certificate .

Sorry I haven't had a moment in front of a computer to look into it yet

On Wed, Jan 29, 2020, 5:47 PM Sowmya Viswanathan notifications@github.com wrote:

There is a dirty dirty hack you can use to disable SSL verification across the platform in Ruby, but I'd have to release a code change. Let me think about how I could do it.

Agreed ! This can be the last last resort.

I'll also re-verify the certs from my end.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pact-foundation/pact_broker-client/issues/63?email_source=notifications&email_token=AAANFDGXJ4O4GWW5W5CWK33RAERBVA5CNFSM4KL3MXL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKGEOEY#issuecomment-579618579, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANFDHC6P4KDLPTJGMN2ZDRAERBVANCNFSM4KL3MXLQ .

bethesque commented 4 years ago

I've never been able to generate a self signed certificate and then use the SSL_CERT_FILE to make the client trust a server running with the self signed certificate - I always attributed it to my lack of understanding though. The only way I've ever gotten it working was to generate a self signed CA, use that to sign a cert, run the server using the child cert, and then connect by setting the CA as the SSL_CERT_FILE.

mefellows commented 4 years ago

Ok this makes sense then. Sowmya, you need to set the SSL_CERT_FILE to the CA public certificate (not the generated certificate). This way, any certificate signed by the CA will be trusted.

On Thu, Jan 30, 2020 at 7:08 AM Beth Skurrie notifications@github.com wrote:

I've never been able to generate a self signed certificate and then use the SSL_CERT_FILE to make the client trust a server running with the self signed certificate - I always attributed it to my lack of understanding though. The only way I've ever gotten it working was to generate a self signed CA, use that to sign a cert, run the server using the child cert, and then connect by setting the CA as the SSL_CERT_FILE.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pact-foundation/pact_broker-client/issues/63?email_source=notifications&email_token=AAANFDAZJW4TY53F5KCMKATRAHO2TA5CNFSM4KL3MXL2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKISDQY#issuecomment-579936707, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAANFDGOFINPGTR66CAEVXLRAHO2TANCNFSM4KL3MXLQ .

-- Matt Fellows www.onegeek.com.au calendly.com/mfellows

bethesque commented 4 years ago

I've released a new version of the pact-cli docker container (0.2.0-1). If you set PACT_DISABLE_SSL_VERIFICATION=true you will get a severe warning printed out, but it should work!

bethesque commented 4 years ago

Ok, I got it working without a separate CA file! I've been trying to do this for years! The key was the extensions in the certificate. This is the combination of extensions that made it work for me. It's possible that some of those are redundant (there are two keyUsage extensions in there) but I don't have time to experiment more unfortunately.

root_ca.add_extension(ef.create_extension("basicConstraints","CA:TRUE",true))
root_ca.add_extension(ef.create_extension("keyUsage","keyCertSign, cRLSign", true))
root_ca.add_extension(ef.create_extension("subjectKeyIdentifier","hash",false))
root_ca.add_extension(ef.create_extension("authorityKeyIdentifier","keyid:always",false))
root_ca.add_extension(ef.create_extension("keyUsage","digitalSignature", true))

So, it is possible to use the same cert file in the server, and set it as the SSL_CERT_FILE to connect. I don't know why yours is not working @sowmyav24, and given it's not on the public internet, I think we've done as much as we can to help diagnose the issue. I'm closing this one, as there is a work around now.

sowmyav24 commented 4 years ago

@bethesque and @mefellows Thanks for the help ! If i am able to debug why it didn't work for the particular cert that I have, I'll post it here.

pavankumar1986 commented 3 years ago

I still see this issue while using pact-1.88.11-win32, I trying can I deploy, it fails. But using docker based implementation with DISABLE_SSL_VERIFICATION works..

mefellows commented 3 years ago

Yes, it’s fixed, the issue is about supplying the custom SSL details.

You’re asking for something different - to disable SSL verification altogether (usually a bad idea, the reasons you likely have invalid certificates is because you work in a corporate environment that values security enough to make it painful to do it a sensible way, which is to use a certificate issued from a trusted public authority).

The change that completely disables SSL verification is only in that docker version - see this commit: https://github.com/pact-foundation/pact-ruby-cli/commit/8375dc8dac9bc7c378f7e230c5b36efce0d3c948

I can see how supply custom certificate details would be a problem on local development, but in CI it should be manageable.

If you really need it, and can’t supply the SSL details any other way, could you please create a separate issue with that request?

pavankumar1986 commented 3 years ago

Hello Matt,

I have the SSL details (the pem file downloaded from chrome). While I am using the docker based implementation by supplying the certificate (via volume) also it fails for same reason. Note: SSL_CERT_FILE=/your/cert.pem curl -v https://your-broker:443, returns success from local (git bash console).

From the pact-broker cli utility after setting the SSL_CERT_FILE to environment variable also it fails. Tried the SSL_CERT_DIR also. Can you please suggest how to make it work from local using pact cli (windows based one)?

Thanks, Pavan

bethesque commented 3 years ago

Try playing around with the docker-compose file mentioned here https://docs.pact.io/pact_broker/advanced_topics/using-tls/#for-non-jvm Work incrementally to make it point towards your own server with your own cert.

pavankumar1986 commented 3 years ago

Thank you beth, will play with docker. But is there a way we can make it work with the windows cli, NOT docker. I am looking for a cli solution.

pavankumar1986 commented 3 years ago

I am using https://github.com/pact-foundation/pact-ruby-standalone/releases on my windows machine. I have set the env variable for ssl_cert_dir but still same issue. I hav tried setting ssl_cert_file also..

mypa commented 3 years ago

If you're also struggling with Docker based PACT CLI to work with your organization's specific certificates, maybe the following steps help:

  1. Copy complete certificate chain (e.g. using Chrome/Firefox) into cer files
  2. Transform them into PEM format: openssl x509 -in $file.cer -out $file.pem -outform PEM
  3. Finally copy them all into one PEM file, e.g. certs/cacert.pem
  4. Start container, e.g. using docker compose up stub

example docker-compose.yml: version: "3.8" services: stub: image: pactfoundation/pact-cli:0.37.2.0 ports:

bethesque commented 3 years ago

Windows kills me! I think there's something wrong/missing with the way the env vars are set in the windows script that starts the ruby process. Have a look at https://github.com/pact-foundation/pact-ruby-standalone/issues/30

styssi commented 3 years ago

you can use curl and disable SSL by adding -k -O at the beginning.

curl -k -O -v -XPUT -H "Content-Type: application/json" \ -d@spec/pacts/a_consumer-a_provider.json \ http://your-pact-broker/pacts/provider/A%20Provider/consumer/A%20Consumer/version/1.0.0+4jvh387gj3