EngineerBetter / concourse-up

Deprecated - used Control Tower instead
https://github.com/EngineerBetter/control-tower
Apache License 2.0
203 stars 28 forks source link

Using AWS managed certificate. (acm) #24

Open engrun opened 6 years ago

engrun commented 6 years ago

We want to use an AWS managed SSL certificate (e.g. generated by ACM), and as far as I can tell, there is no way for us to get our hands on this certificate's private key. Do you know if it's possible to run concourse-up with a ACM requested certificate (not imported)? If so, how?

peterellisjones commented 6 years ago

Hi Engrun,

This is not currently implemented and is something we'd like to implement at some point. However you can currently do this manually by following these steps:

cheers,

Pete

engrun commented 6 years ago

Thanks for the tip. We had been discussing the same approach. However, running concourse-up help deploy no such flag, custom-domain, is listed?

And now concourse-up has generated a self-signed certificate. We probably need to disable this. I guess the loadbalancer will not accept the self-signed certificate?

engrun commented 6 years ago

I have already run with the domain flag. I guess that's what you meant.

peterellisjones commented 6 years ago

oops yeah domain not custom-domain

The load balancer can be used with a certificate you will need to manually generate in AWS Certificate Manager

engrun commented 6 years ago

Yes, I understand I have to generate the certificate and use that with the ELB.

However, when running concourse-up, a self-signed certificate is generated. (not by AWS). When pointing the ELB to the webnode, the webnode has a certificate that is not "trusted". My question is whether the ELB will accept this self-signed certificate. I guess I will find out :)

My initial thought was to perhaps terminate SSL at the ELB

peterellisjones commented 6 years ago

Yes you will need to terminate SSL on the ELB and forward unencrypted traffic to the Concourse web node on port 80

On 9 November 2017 at 18:15:40, Rune Engseth (notifications@github.com) wrote:

Yes, I understand I have to generate the certificate and use that with the ELB.

However, when running concourse-up, a self-signed certificate is generated. (not by AWS). When pointing the ELB to the webnode, the webnode has a certificate that is not "trusted". My question is whether the ELB will accept this self-signed certificate. I guess I will find out :)

My initial thought was to perhaps terminate SSL at the ELB

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

engrun commented 6 years ago

hi We have tried to configure this now.
You say we should forward the ELB for concourse-web on port 80. This does not work as concourse will redirect to https. Proved by (request webnode on port 80 directly) curl -H 'Host: ci.example.com' -H 'X-Forwarded-Proto: https' 54.x.x.x.

This yields

HTTP/1.1 301 Moved Permanently
Location: https://ci.example.com/
engrun commented 6 years ago

To sum up. I think the problem is that concourse-up enforces a self-signed certificate, and cannot, as of now, be configured to use port 80.

peterellisjones commented 6 years ago

That's odd — so to clarify, you are accessing the ELB using HTTPS (eg https://ci.example.com/), and you are getting a redirect?

peterellisjones commented 6 years ago

It looks like Concourse always redirects to https when a cert is provided

https://github.com/concourse/atc/blob/01e6614e28056eb101fe7340ac27bd97e65badf7/atccmd/command.go#L414-L421

https://github.com/concourse/atc/blob/01e6614e28056eb101fe7340ac27bd97e65badf7/atccmd/command.go#L1124-L1126

So you may have to re-deploy the concourse using BOSH manually with the TLS bind port set to null, or by removing the tls cert and key from the BOSH manifest

http://bosh.io/jobs/atc?source=github.com/concourse/concourse&version=3.6.0#p=tls_bind_port

https://github.com/concourse/concourse/blob/master/jobs/atc/templates/atc_ctl.erb#L101-L104

engrun commented 6 years ago

That's odd — so to clarify, you are accessing the ELB using HTTPS (eg https://ci.example.com/), and you are getting a redirect?

-> https: yes, getting a redirect: yes

So you may have to re-deploy the concourse using BOSH manually

This is a path we are not going to take. (One of the "weaknesses" of Concourse is failing to provide a detailed install instructions for the most common cloud providers). And btw, that's why we see concourse-up as a very nice tool!

However I assumed that, when forwarding the ELB to a HTTPS endpoint (webnode in our case) with a self-signed cert, the ELB would not allow this. But, we tried, and it works. That is, without terminating SSL at the ELB So Browser -> HTTPS -> ELB -> HTTPS -> webnode

JasonMorgan commented 6 years ago

@engrun can you display your ELB configuration? Specifically I'm curious if you had to tell it to trust the self signed cert or if just ignored SSL errors by default.

JasonMorgan commented 6 years ago

Just as an addition to my last note, there is no requirement to give the ELB the self signed cert. This configuration works like a charm.

JasonMorgan commented 6 years ago

That being said once I had the ELB running I wasn't able to intercept containers. Is anyone else running into this?

walked commented 5 years ago

@JasonMorgan I'm about to go down this path myself; did you ever get intercept working? I saw this:

If you're using an AWS ELB, you have to make sure that the protocol forwarding to concourse:web on port 8080 is ssl and not https.

https://github.com/concourse/concourse/issues/1342

Curious if you got there with an ELB; just getting my pre-planning ducks in a row before I start doing all my deployment work.