subdomain www.jupiterbroadcasting.com certificate error

elreydetoda commented 2 years ago

Because @ironicbadger said the strict enforcement during this: https://github.com/JupiterBroadcasting/jupiterbroadcasting.com/issues/292#issuecomment-1216730781 (which is good to have)

There is a certificate error on the www. subdomain:

Should be fixable by using cloudflare pages (mentioned here):

This seems to be the only thing I can find about it, they're referring to Cloudflare Page rules IIRC: https://community.cloudflare.com/t/redirect-www-subdomain-to-apex-at-the-dns-level/226074/2

reason why happening is explained here.

or either choosing flexible (which technically lowers security, so the aforementioned fix is prefered): https://github.com/JupiterBroadcasting/jupiterbroadcasting.com/issues/292#issuecomment-1216730781

ironicbadger commented 2 years ago

We've moved the site to www and the root will shortly redirect there too.

https://github.com/JupiterBroadcasting/infra/pull/6

ironicbadger commented 2 years ago

We have decided not to do anything with the domain root at all. All traffic is going to www and @ is now defunct.

elreydetoda commented 2 years ago

We have decided not to do anything with the domain root at all. All traffic is going to www and @ is now defunct.

Since that means the APEX domain is no longer getting used now, can we not use CNAME flattening now?

ironicbadger commented 2 years ago

We have decided not to do anything with the domain root at all. All traffic is going to www and @ is now defunct.

Since that means the APEX domain is no longer getting used now, can we not use CNAME flattening now?

Nope!

elreydetoda commented 2 years ago

We have decided not to do anything with the domain root at all. All traffic is going to www and @ is now defunct.

Since that means the APEX domain is no longer getting used now, can we not use CNAME flattening now?

Nope!

I think this is because you were following the paid account steps, but if you'd done just a cname record with the proxied DNS from apex domain -> www. it should have worked (instead of trying to access the "Flatten CNAME at root" option (which is only good for grey cloud resolutions according to the docs)).

This option is useful for DNS-only (unproxied) CNAME records since proxied records are flattened by default (as they return Cloudflare edge IPv4 and IPv6 addresses).

So...this issue was driving me insane not knowing what was going on :sweat_smile:... (honestly still haven't been able to validate it, but I think I understand why it wasn't working)

typically I've done cname flattening with GH pages & cloudflare...so I figured that you could just point it to the domain and it'd handle all the redirection for you...but transparently I guess GH is also pulling a certificate for your apex domain as well and adding a redirect w/nginx...

I figured this out, because I help setup this site: https://www.climaterefugeestories.com/, and I also run my personal blog at blog.elreydetoda.site. They're both fronted by cloudflare (w/proxied DNS (orange cloud)), but only the former is hosted on GH. The later is hosted on Pikapods (awesome hosting place BTW (sans this one issue)), and I disabled my proxied DNS so I could get the certificate that they're generating, when I put in an alternative domain name in their configuration (ran this command: openssl s_client -connect blog.elreydetoda.site:443 <<< '' 2>/dev/null | openssl x509 -noout -text to get the following text):

blog.elreydetoda.site (grey cloud)

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            9d:61:3e:87:f3:5d:3b:76:9a:5a:a7:9d:e5:08:c3:e6
        Signature Algorithm: ecdsa-with-SHA384
        Issuer: C = AT, O = ZeroSSL, CN = ZeroSSL ECC Domain Secure Site CA
        Validity
            Not Before: Jul 22 00:00:00 2022 GMT
            Not After : Oct 20 23:59:59 2022 GMT
        Subject: CN = blog.elreydetoda.site
        Subject Public Key Info:
......... (truncated) ............
            X509v3 Subject Alternative Name: 
                DNS:blog.elreydetoda.site
    Signature Algorithm: ecdsa-with-SHA384
......... (truncated) ............

The important note is the SAN (subject alternative name), which pikapod's provider only requests the name that you specify. Compared to cloudflare's cert gets it for both your apex & wildcard:

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            0b:db:6a:e1:5c:a6:5e:88:14:86:48:5c:21:42:6e:9e
        Signature Algorithm: ecdsa-with-SHA256
        Issuer: C = US, O = "Cloudflare, Inc.", CN = Cloudflare Inc ECC CA-3
        Validity
            Not Before: May 17 00:00:00 2022 GMT
            Not After : May 17 23:59:59 2023 GMT
        Subject: C = US, ST = California, L = San Francisco, O = "Cloudflare, Inc.", CN = sni.cloudflaressl.com
......... (truncated) ............
            X509v3 Subject Alternative Name: 
                DNS:sni.cloudflaressl.com, DNS:elreydetoda.site, DNS:*.elreydetoda.site
......... (truncated) ............

and I'm guessing that GH pages also does the same thing (with getting the APEX & wildcard cert), so that's why GH pages is able to present a cert for your apex domain...

Validated they add a 301 redirect via NGINX as well:

So, that's my thought as to why this wasn't working for us...we'd have to have traefik get a cert for both the apex & www domain (which it is now), and then get a redirect from www -> apex domain.

Apparently there is supposed to be some configuration that could handle this, but I'm not well versed enough in traefik to understand how :sweat_smile:

PR: https://github.com/traefik/traefik/pull/3403 Flag (search for: CNAME flatten): https://doc.traefik.io/traefik/reference/static-configuration/cli/

An alternative solution, if you wanted to keep it IaC and were using terrform :upside_down_face:, would be to use the Cloudflare's page rules (mentioned it in matrix before), but I just setup a redirect for my blog's apex domain using them to test it out:

ironicbadger commented 2 years ago

Putting all JB DNS in Terraform sounds like a great idea. I made a start but never properly finished here.

Thank you for your incredibly detailed note above I will reach out to you shortly to implement.

elreydetoda commented 2 years ago

Putting all JB DNS in Terraform sounds like a great idea. I made a start but never properly finished here.

Oh, I didn't know you felt that way 😅 I thought that you just wanted to ansible all the things instead of using terraform. I'd be happy to help contribute with that plan (if that's ok 🙃)

(Longer term topic)

I don't know how JB currently does server deployment, but there's actually a pretty cool way to tie in terraform deployed servers with ansible 😁. Then you can just reference those deployed servers with terraform data references for the cloudflare config 😁

Thank you for your incredibly detailed note above I will reach out to you shortly to implement.

No problem, and no rush. I'm here to try and help out for the long term 😁 you've got it working now, so we can postulate about this stuff whenever you have some free time 🙂

ironicbadger commented 2 years ago

I don't see the value personally of deploying the servers via terraform here. We deploy this server once and forget about it.

By having the config in terraform obviously infra as code principles apply and things are more repeatable. However the upsides are also a huge downside. Fat fingers could easily delete the server too.

DNS is one thing. It's stateless. But the server itself of which we only have a handful probably derives little value from being fully terraform'd.

There's not much to do to productionise the DNS code I already wrote tbh.

See this post for how I do secret lookup. https://blog.ktz.me/store-terraform-secrets-in-yaml-files-with-yamldecode/

elreydetoda commented 2 years ago

I don't see the value personally of deploying the servers via terraform here.

IMO, there are a few reason:

bus factor/DR (disaster recovery)
- if the worst scenario happened of you were no longer at JB, and (I'd hope this never happened, but just postulating) all the servers were deleted from linode. Is there documentation on which instances you chose? Could anyone in JB be able to re-deploy all the servers that they/you created? If so, how long will it take them? Hours, days, etc. (with terraform it'd be minutes :upside_down_face: )
templating - while for you deploying a server might be trivial, there could be a time when you're on vacation and you'd like to not have to deal with a server going down (i.e. the main JB website) or that's accidentally deleted from the web-ui. Then no one has to call you because it's already defined in terraform code + GH actions. They don't have to ask you what the specs where, because terraform has that, and then it could be automatically provisioned w/ansible after terraform's deployment
uniformity/consistency - while chris talked about it being weird having production changes all happen inside GH action (in Office hours 11) via web-ui's. This would bring that to the next level, now servers would be defined via GitHub as well, and that means it'd give others a chance to review (when a server is being deployed) and give feedback (if wanted) via the PR process. Maybe someone forgets to add an ssh key by default, or maybe they don't add a tag/label to a server, etc...
- if you always give something a dns name after you deploy it, then you could make sure someone adds a cloudflare dns record at the same time as them deploying a server. Both changes could happen simultaneously (and whatever other services you touch when you deploy a server).
clarity/security/logs - allowing all this to be in the open would also viewable/auditable for any and everyone (which could also be a downside IMO, but I think it's more of an upside overall). People wouldn't have to ask you what are the specs for a machine they're deploying something on, or people could try and reproduce something in their linode accounts if you're having an issue.
- if someone asks about "have you tried xyz" and everything is deployed via IaC (server + provisioning) then you can say "ya, I tried that and you can see how I configured everything with this PR". Then they might have some feedback of something like "oh, ya you need to make the instance larger" or "ah, ya you need to also set this environment variable" allowing complete transparency when you're doing something
- If someone in the community is a SME (subject matter expert) about a specific thing and they want to contribute to JB by deploying a server specifically tuned in a specific way (i.e. matrix (since it's been such a beast to manage)), then they'd be able to create an issue (to get your approval), and then submit a PR. They don't even have to have access to your linode account, and

We deploy this server once and forget about it.

I'd honestly argue that's even more of a reason to do terraform deployments for IaC. Maybe there was a specific reason why you went up an instance size, then you could document that along with the code to actually deploy it. Or if you're on vacation and someone needs to increase the size, (in AWS they can do that without bringing the server down, so I'm going to make the same assumption here but don't personally know how linode handles it) so all they have to do is change the size in the code, look at the plan, and then let GH actions do their thing.

For example, I like using jitsi but since I'm a bit more privacy/security concision than the average individual. So, I created a combination of terraform + ansible to deploy a jitsi instance on demand via DigitalOcean ( tried to migrate to Linode's terraform, but it wasn't as fully featured (IMO) back then and I created issues on their GH about it which has made it nicer (probably will switch when I have time :upside_down_face: )). Not only do I get a cost savings, because I also have a step to delete the server, but because it's something I only do like once every 2 weeks I don't have to worry about if the size of the server is right or not. If there were specific configurations I needed during deployment I'd be able to document them and hold on to why I added those configuration option.

However the upsides are also a huge downside. Fat fingers could easily delete the server too.

In theory (yes, I know this is the real world and things happen :sweat_smile:, but could also happen via a web-ui as well ), this shouldn't happen because we'd integrate a step where we run terraform plan before the PR is merged so we can see what changes would happen. That way there shouldn't be any fat fingering, and everything that happens should be expected.

DNS is one thing. It's stateless. But the server itself of which we only have a handful probably derives little value from being fully terraform'd.

I'm not going to argue with there being some complexities to a production terraform deployment, and there will probably esoteric difficulties here and there + quite a bit of work to get everything fully automated (especially only using GH (GitLab has some nicities like managing your terraform state).

At the end of the day it's whatever you're most comfortable with and however you can get your job done the best, but personally I think the explicitness/transparency adds more value than detracts. (but that's just my personal opinion :upside_down_face: ).

There's not much to do to productionise the DNS code I already wrote tbh.

Cool! I'll take a look at that when I get a chance :grin:

See this post for how I do secret lookup. https://blog.ktz.me/store-terraform-secrets-in-yaml-files-with-yamldecode/

So, I'm guessing this is for reading in ansible-vault information? Or else where are those files stored when the apply happens? If it's not ansible vault stuff, is there a reason why they can't be environment variables either in the repo's settings or via an .envrc file + direnv instead of reading from files?

JupiterBroadcasting / jupiterbroadcasting.com

subdomain www.jupiterbroadcasting.com certificate error #358