Move away from Dreamhost to hosting that gives us full root access

sideshowbarker commented 7 years ago

Some context in the following IRC discussions with @annevk

A few problems we have with hosting through Dreamhost:

Their IPv6 setup for .whatwg.org continues to be seriously broken—though Dreamhost claims there actually is no problem. But despite their claims I am certain they have a chronic consistent problem that causes IPv6 connections to appear like they’re basically being tarpitted. So on every machine from which I need to get to .whatwg.org I have to edit the /etc/hosts file to force the 75.119.197.251 IPv4 address to be used instead. Among other things I need to do that for all places where I run an instance of the HTML checker, because the checker needs to fetch content from wiki.whatwg.org at start time.
We recently had a more acute problem—with no workaround—when Dreamhost broke TLS certificate validation for *.html.org resources for at least a full day. It’s not clear how they broke it nor how they fixed it but @annevk says they claimed there actually was no problem.
Today @annevk noted that “DreamHost's SSH instructions are basically "trust the network"… They move the server and there's no place to find the new public key securely”
https://lists.whatwg.org/pipermail/whatwg-whatwg.org continues to be broken and apparently unfixable as long as we continue hosting at Dreamhost

The above is not even an exhaustive list—but I hope it’s enough evidence to make clear it’s time we seriously consider a move to hosting at DigitalOcean or some other provider that gives us control over the environment and ability to prevent problems like the above and fix them when needed.

Hixie commented 7 years ago

The main advantage of Dreamhost is that y'all don't have to pay anything for us to host there, since I'm eating the cost as part of hosting my other servers. :-)

foolip commented 7 years ago

I don't particularly care if we use Dreamhost or not, but we should find a way to fix these problems. That links to the old list archives are broken is my least favorite thing, although one can't entirely blame Dreamhost for that since we didn't have to enable HSTS.

As far as cost goes, I'm sure the cost of any solution would be moderate and could be sponsored by any number of people/organizations.

annevk commented 7 years ago

I think we should go ahead with this. I don't think we should move everything, especially not the more complicated PHP sites, but everything under *.spec.whatwg.org would be a good start. We can't have our standards not be reliable and taken down by DreamHost without it being clear how to solve the DOS problem they claim to see.

domenic commented 7 years ago

+1, although I think moving everything might also be doable.

domenic commented 7 years ago

Here is a list of "everything" in terms of DNS records:

A: whatwg.org, blog, c, forums, ftp, help, images, lists, mail, mailboxes, n, ssh, svn, webmail, wiki, www.blog, www.mailboxes, www.webmail, www
CNAME: build
MX: whatwg.org, mail
TXT: _domainkey

Let me open a new issue on sunsetting some of the older stuff which would make a move easier.

domenic commented 7 years ago

@annevk pointed out in IRC this is missing some subdomains like *.spec.whatwg.org and validator.whatwg.org. So this auto-scanning I was using to try to find everything seems to be a bust, and we need @Hixie to help us (ideally by exporting the current set of DNS records in BIND format)

foolip commented 7 years ago

I've set up a digitalocean team with a test server. Invites have been sent out.

snuggs commented 7 years ago

@domenic @annevk @foolip without being sent to the canonical answer tweet for "What's the status on this?". Is there any way to get visibility into this? Have access to various hosting companies inclusive of Digital Ocean who would indeed love to help with #GotRoot?. Will be speaking with DO as well next week as they are giving some talks at my meetup. Would love to help expedite the process in any way I can. These types of errors are not a good look.

Please advise

annevk commented 7 years ago

Heya, we've been slowly moving things to Digital Ocean. https://github.com/whatwg/misc-server and https://github.com/whatwg/whatwg.org have most of this activity related to this (and status issues), though some specifications also required some changes.

We haven't reached out to Digital Ocean for some kind of free-for-organizations-type-plan though. Might be interesting.

foolip commented 7 years ago

https://github.com/whatwg/misc-server/issues/7 is the issue tracking the remaining domains.

If someone wants to get us free hosting with DO, that'd be great. It isn't costing very much right now though.

snuggs commented 7 years ago

@foolip @annevk have reached out and got response from my DO connects. Am currently in San Francisco and we are scheduled to link up next weekend when I am back in NYC. Will also make reference to https://github.com/whatwg/misc-server/issues/7

No promises but as my mother always says "Sometimes you have not because you ask not. Worse someone can say is 'NO(t right now)'" ;-)

Will keep thread posted...

foolip commented 7 years ago

Thanks @snuggs!

snuggs commented 6 years ago

@foolip @annevk @domenic Got the thumbs up! Asked one of DO directors at my last meetup. Will probably need to go visit them and take someone to lunch when I get back from Paris next week. Am speaking at a conference in N.O. then back to NYC on the 19th. Basically said "love to Snuggs" and asked scope of project. I said "We have a few OSS domains that need root and sponsored hosting" but could wildly be oversimplifying. Please advise and i shall make it happen.

foolip commented 6 years ago

@snuggs, we already have an account that's running a few servers, so I guess the next step is to stop paying :)

snuggs commented 6 years ago

Word @foolip. Pretty sure I can finesse the situation but would rather do face to face with DO in a couple weeks. I believe can get the process started in the meantime. Thanks for your patience. Looking like it's going to work out for us. Will update the thread accordingly.

foolip commented 6 years ago

It's fine to just keep paying as well, so no worries.

snuggs commented 6 years ago

@foolip copy that. We have options. If we can we can. If we can't we're still good. 💪

snuggs commented 6 years ago

@foolip @domenic @annevk I reviewed our account with Digital Ocean at our conference this past week as we intended. We got the green light. Amount of subdomains no problem apparently. And using minimal "droplets 💧 💦 " I believe they are called. I have sent @foolip a thorough email along with introduction to my 🔌 plug with DO. Please check and confirm receipt. I used the address from your personal site. You should be receiving one from the address on my profile.

Done and done! 💪 🙏

foolip commented 6 years ago

Thanks @snuggs, I've got the email and will respond in a somewhat timely manner :)

annevk commented 6 years ago

Closing this per https://github.com/whatwg/whatwg.org/commit/ee73a964f84f1f37288a7765f7b04976fd70237f. The remainder is tracked by https://github.com/whatwg/misc-server/issues/7. Thanks everyone!

snuggs commented 6 years ago

Sweet @annevk. @foolip has been moving the ball forward i've noticed in email conversation and I don't think needs anything else on my end. Awesome!

whatwg / meta

Move away from Dreamhost to hosting that gives us full root access #8