Open daniloc opened 5 hours ago
I'll start by ultimately saying: don't worry about the builds! 😊 That's what we're here for – to monitor things, fix issues as they arise, and get PRs merged that are ready to go live. Hopefully builds aren't getting in the way of you writing content and putting up PRs!
If you're having issues running the site locally, let's definitely get to the bottom of it! Reach out in Slack and we'll get you taken care of.
But generally speaking we're always around and monitoring the build situation. If a build is extremely time-sensitive, definitely shoot a message in Slack and someone can restart a build if it fails. Otherwise, we'll usually get to it within an hour or two. (And you can definitely have a Vercel seat if it's helpful for you - we just don't hand them out to everyone by default due to cost.)
Vercel builds fails mostly due to memory spikes during the build process. When this happens, this is all you see in Vercel:
We've spent a lot of time debugging and making optimizations – and it's still ongoing – and we've been in touch with Vercel, though they've been mostly unhelpful. Our issue would ultimately be fixed by a memory increase, though they won't give it to us unless we entertain upgrading to enterprise. Our monthly bill is north of $500/mo as it stands now – more than half of which is just seats. Our bill would multiply by a significant factor if we went the enterprise route, which I think we can all agree is absurd for a static website.
In working with Vercel support, I've found them to be well-intended though mostly unhelpful ("Try upgrading to Node 20") or suggesting things we've already tried – and extremely slow to respond at that (last reply took 4 days).
We've also explored other hosting options (like Amplify) where build consistently work (and with even less RAM than Vercel, mind you), but there is a security risk in that, out of the box, we can't require approvals for external contributors. (You could theoretically print out all of our environment variables, for example.) Eli is looking at workarounds for this.
This isn't to say 100% of the issues are on Vercel. Sometimes builds will fail for other reasons like:
master
merged into it to get the latest updates. This is a mostly recent issue, as we've been going through more and more renames, something that was a rare occurrence in the past. We'll find a better process for this at some point. It just hasn't been top of the priority list yet.Some of these we fix as they come up, some we haven't as they're rare and not worth fixing compared to other priorities.
When running the site locally, there are some steps that are omitted like building the careers section as well as the merch store – areas of the site that regular contributors don't need. This actually greatly speeds up starting the local development server.
To build the full site locally, you just need API keys. This would likely be why you're getting an Ashby failure or having issues building locally. (I've personally never seen that Ashby error before – but I also run with many more API keys.)
Generally speaking as we see issues with running or building the site, we put in a fix if it's warranted. But we also don't know of all issues everyone else has with running the site locally, so definitely give a shout if you're having specific problems. We'll at least be able to point you in the right direction – if not give you the fix immediately – and it will It definitely save you hours of trying to reverse engineer 3+ years of code. We've got lots of knowledge here - happy to share it, just ask!
Thank you for the backup!
Hopefully builds aren't getting in the way of you writing content and putting up PRs!
Unfortunately, they have, enough I felt compelled to raise this issue and re-prioritize things if it would be helpful. I should emphasize that I'm speaking specifically of builds that yield deploy previews, as:
.mdx
contentyarn start
for all the reasons you mentioned, so in cases of ambiguity, it's nice to see exactly what will ship before a mergeA single flat .md
file is straightforward enough under status quo.
But wading into the docs end of the house this week has been made thornier by the unpredictable, elliptical orbits of the deploy previews. In one particular case, I'd put down serious money I could have doubled my output for a day or two had they worked reliably.
I think our approach to docs and content folded in as first-class citizens of the .com site is infinitely better than what's typical, so I'm happy to sustain the long-term complexity lift it demands.
But man, there's some shin-bangers lurking there.
Let's just start with inviting you to Vercel so you can peep your own builds. Sent you an invite.
I volunteer as tribute.
I will dedicate immediate bandwidth to address this but I'm starting an issue before diving headlong into the problem:
It is hard to work on the static site
Local builds take awhile, Vercel preview builds take eons, and transient errors in both domains compound with the build times to bleed away time, energy and forward momentum.
Example case
yarn build
./careers
path:Not ideal, and over time, corrodes the leverage of all the salaries going into site content.
What can we do?
All of my suggestions are naive, I'm just starting the convo:
If there's stuff backlogged and you need me to be your developer experience donkey to pull it out of the mud, you can have my unlimited and immediate bandwidth to accomplish this! The compound returns fully justify the side quest imo.
Thank you for your time and generous indulgence!