Migrate hosting to Cloudflare

kimadactyl commented 2 years ago

User story

As a developer, I want to know that I won't have any hosting issues and can leave this unattended for the forseeable, in order to reduce "suprise" work and dependancy on unreliable/hacky methods.

Acceptance criteria

[x] Migrate site to Cloudflare
- [x] Builds on commit to main
- [x] branch preview
- [x] #325
- [x] stop netlify building site
[x] Ensure that the emailer function works and/or migrate this to cloudflare too
[x] Update README and other refs to netlify URL
[x] Verify config var values with GI (send from email, admin email, canonical etc) and set them accordingly

Implementation notes & questions

This follows on from #287

I think we should move this to Cloudflare. The free package is orders of magnitude better than netlify and has:

Unlimited build minutes, ensureing that we don't get DoSd on that bare endpoint, and meaning we can rebuild every hour with no issues - 300 minutes might be a real issue for us esp now that endpoint is visible in our github
Built in cron - remove reliance on GitHub Actions hacks that apparently go down after 60 days, and make this transparent
Proper multi auth login - this has been a real issue to date
Better stats
Free functions if we need them
More generous (unlimited?) bandwidth limit.
I generally find overall performance better for me on CF vs Netlify sites - they just seem to load faster

I know there were some political concerns but TSN have decided we are collectively happy with CF now they have in fact, booted the bad site, esp given it's not like other providers are queuing up to say that they wouldn't allow that kind of content. I personally don't think it's worth us thinking about it any further than that but open to other opinions. This just solves several very real problems for us, and prevents several future problems, and we are short on time to get this done as it is. It's also prefigurative in that we can fully test it works for the next site, and is a long term free solution for this.

That said - all the above can change so this seems like a good time to configure this with Terraform too.

Implementation plan

katjam commented 2 years ago

I am tempted not to prioritise this if our current set up is not broken. All of these CI / hosting services do this type of offer and then cut it off suddenly. They are competing for each other's business. Over the last 5 or 6 years I think I've been through like 15 services and its a very familiar pattern.

We probably do want to choose a favourite (which might be cloudflare) and use is for all studio projects unless a good reason to go elsewhere.

Note: The service should also probably always be in client account rather than GFSC unless they are paying support contract. So we'd need GI to down Netlify and sign-up for this instead. I've not checked if they are running anything else on there - but would be good to help them migrate everything if they are.

erbridge commented 2 years ago

I agree with you @katjam.

I also think at this late stage, switching host to a fresh setup is going to cause more issues than it prevents. I think it would be different if we already had reproducible "deploy me to X" infrastructure, but there's a lot of risk of misconfiguration subtleties which we've already worked out for the current host that I don't see the value in doing anything about right now.

kimadactyl commented 2 years ago

The thing is the cutoff date for the current service was 30th Sept so I am sort of expecting any problems to happen from today.

GI already have cloudflare set up I think yes.

I do really feel like the first bullet point is a very serious and real security issue especially given the repo is open source and am suprised higher weight isn't being given to that. Also feel it's a lot easier to switch over now than later.

To me this is an ee / vvv ticket - i feel like the value in not having to worry about these things far outweighs a few hours potential updating the config, which will only make us more resiliant anyway.

kimadactyl commented 2 years ago

Just to note that we are nearly at monthly capacity and if we hit it their website will stop building too (which is using the same netlify account).

erbridge commented 2 years ago

@kimadactyl I've disabled the workflow now.

katjam commented 2 years ago

Ok - let's just get their Cloudflare creds annd move it.

Does Cloudflare also allow for Lambdas - or will be leave the Join Us mailer on netlify?

aaaaargZombies commented 2 years ago

Notes for the next person (maybe me)

I've got the site building at transdimension.pages.dev. The build environment variables are set to this in both netlify and cloudflare.

Variable name
Value

CANONICAL_URL
http://transdimension.pages.dev | https://transdimension.netlify.app/ respectively

EMAIL_ADMIN
alfie@gfsc.studio

EMAIL_FROM
enquiries@trans-dimension.com

EMAIL_INFO_TRANS_DIM
alfie@gfsc.studio

JOIN_US_FUNCTION_URL
https://transdimension.netlify.app/.netlify/functions/transDimMailer

PLACECAL_API
https://placecal.org/api/v1/graphql

SMTP_HOST
smtp.mailersend.net

SMTP_PASSWORD
#################### secret

SMTP_REQUIRE_TLS
true

SMTP_SECURE
true

SMTP_USERNAME
#################### secret

When submitting the form on https://transdimension.netlify.app/join-us/ we're getting a 502 - bad gateway When submitting the form on https://transdimension.pages.dev/join-us/ we're getting a CORS missing allow origin and 502 - bad gateway

It's possible to look at logs in Netlify by going to app.netlify.com/sites/transdimension/functions/transDimMailer

katjam commented 2 years ago

These vars are obviously temp values. I have put an AC item to address this before we release / approve this ticket.

kimadactyl commented 2 years ago

Can I suggest we just use a service like Formspree if this issue persists?

aaaaargZombies commented 2 years ago

So cloudflare is hitting the function but it it's still grumpy about the CORS interms of a repsonse. But it's trying to run the function with the data so I guess this is two issues.

If we get a bad cors response the ELM app will say Your message failed to send. Please try again. even if it takes the data and does what we want with it.
It's not sending an email.

The error at the bottom of the page suggests theres an error with the TLS on the email side of things but I need to look into it more.


Oct 13, 12:32:14 PM: 7150c61b INFO       '<strong>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       'Postcode: </strong><em>Not answered</em>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</p>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<p style="font-family: Verdana, sans-serif; font-size: 15px;">\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<strong>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       "I'd like a ring back: </strong>No\n" +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</p>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<p style="font-family: Verdana, sans-serif; font-size: 15px;">\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<strong>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       "I'd like more information: </strong>No\n" +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</p>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<p style="font-family: Verdana, sans-serif; font-size: 15px;">\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<strong>Message:</strong>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</p>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<p style="font-family: Verdana, sans-serif; font-size: 15px;">\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       'from cloudflare ????\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</p>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</div>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<div id="message-footer" class="banner" style="display: block; background-color: #53C3FF; padding: 10px 10px 10px 10px; color: #040F39; font-family: Courier, monotype, sans-serif; font-size: 12px;">\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '<p style="text-align: center">Reply to this email to reply to the sender.</p>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</div>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</div>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</div>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</body>\n' +
Oct 13, 12:32:14 PM: 7150c61b INFO       '</html>'
Oct 13, 12:32:14 PM: 7150c61b INFO   }
Oct 13, 12:32:14 PM: 7150c61b INFO   catch
Oct 13, 12:32:14 PM: 7150c61b INFO   [Error: 140486543583168:error:1408F10B:SSL routines:ssl3_get_record:wrong version number:../deps/openssl/openssl/ssl/record/ssl3_record.c:331:
Oct 13, 12:32:14 PM: 7150c61b INFO   ] {
Oct 13, 12:32:14 PM: 7150c61b INFO     library: 'SSL routines',
Oct 13, 12:32:14 PM: 7150c61b INFO     function: 'ssl3_get_record',
Oct 13, 12:32:14 PM: 7150c61b INFO     reason: 'wrong version number',
Oct 13, 12:32:14 PM: 7150c61b INFO     code: 'ESOCKET',
Oct 13, 12:32:14 PM: 7150c61b INFO     command: 'CONN'
Oct 13, 12:32:14 PM: 7150c61b INFO   }
Oct 13, 12:32:14 PM: 7150c61b Duration: 191.74 ms   Memory Usage: 70 MB

kimadactyl commented 2 years ago

yeah this sounds messy. propose setting up formspree for now and then opening a new ticket for this so we can get this ticket closed?

aaaaargZombies commented 2 years ago

Related #318

geeksforsocialchange / the-trans-dimension