RFC: Kill ./install and ./configure

[aendra-rininsland]

These are both BASH scripts that, while not terribly complex, are annoying to keep working in both Zsh and BASH, break the language idioms of Starter Kit, haven't been touched in ages and are overly complex from a UX perspective. They also do some pretty weird things with Git that are just wholly unnecessary (see, for example, all the .netrc instructions in README).

There are also better, newer approaches we can leverage with little difficulty.

Solutions:

• [ ] Use GitHub's new "Template repo" feature.
  • Pros:
  • This expects users to populate config/* themselves, which might not be a bad thing as it implicitly requires them to replace all the boilerplate in those files (something that still needs to be done in the current implementation but tends to be the last thing anyone ever does, possibly leading to boilerplate lipsum being shipped)
  • No additional tooling needed whatsoever
  • Cons:
  • Requires a degree of familiarity with the GitHub user interface; cloning repos
  • Easy to forget to switch to v2 branch, though this can be improved by making v2 the primary branch (which we'll be doing pretty quickly anyway)
  • Less control over scaffolding
• [ ] Write a quick NodeJS project that does what install/configure already do, ostensibly via npx @financial-times/sk [path] [flags]
  • Pros:
  • npx caches dependencies so after the initial usage it'll be faster
  • npx will always grab the latest version of the scaffolder
  • More control over scaffolding process
  • Can ask user questions
  • Allows wizard-style configuration of the config/* files
  • Cons:
  • Extra codebase to manage
  • Enables passing --no-optional to npm by default, enabling small node_modules
  • Starter Kit is primarily used by experienced developers at this point and this level of hand-holding is probably unnecessary; it might be helpful for onboarding the data reporters on the team but they're also fairly familiar with Git already.

If you could please respond with your preference of these two approaches, I can move forward as one of the few projects I'll have time for in the six days I'm in the office this month — thanks!

[tlfrd]

I prefer the first solution

[maxharlow]

In my mind the confused part of Starter Kit is that it mixes and conflates two different things -- boilerplate that's expected to change, and various build and setup stuff that isn't. I imagine this is due to the way Starter Kit has evolved rather than by design -- the creation of g-components was a great step forward in spitting the two apart, but the stuff that's left in this repository is still a mish mash of both.

The issue you mention about shipping Ipsum is symptomatic. There's no clear place for things such as type checking and validation of the config. It also means that when Starter Kit is updated, if you want to bring those updates into a project you're working on it's a tedious manual diff and copy-paste process. Worse, as Starter Kit can't be properly versioned in its current state, so trying to understand a Starter Kit project build means mentally comparing the state of Starter Kit at the point the project was created with the file you're looking at.

In my view the solution would be to transform Starter Kit into a static site builder that runs on the command line. It'd include everything that isn't boilerplate in this repo, including things like config default values, but not the boilerplate Ipsum stuff. It would have proper versions published to NPM. Running it would do everything necessary to produce a dist directory.

Then you'd separately have a new project generator that would produce something quite minimal. That would include a package.json with g-components and the new Starter Kit as versioned dependencies, and add a script to build via Starter Kit. The rest would just be boilerplate example files that we expect to modify. It would generate a config file, but only including required values as examples, as anything optional would fall back to defaults specified in Starter Kit. There'd also be an app.js with our Ipsum.

Which is I suppose a longer way of saying I prefer the second option

[aendra-rininsland]

@maxharlow This is a super good insight, Callum had thought way back in the day that we should just use create-react-app and lean entirely on that for our build-tooling.* Honestly, I'd love to do that and make SK just the most basic of scaffolders, it would be one more complex bit of code that we simply don't have to maintain.

I actually investigated that idea a bit and here were my conclusions:

• create-react-app (hereafter CRA)
  • This is kind of industry best practice at the moment but the problem with it is bolting on things like Sass is a little difficult. Just putting my biases out there, I would looouurrrrvvveee to kill Sass at some point. If we could just write plain CSS with variables (now supported in pretty much everything), it would probably shave 4-8 seconds off each page refresh while developing. I haven't really pursued this route because of Origami, but depending on what they do with spec v2 and what we do with g-components, we might not even need Sass. SSR-style static generation is generally done via react-snap these days, which I use near exactly in this PR.
• Zeit NextJS
  • Quite a good candidate and has excellent SSR static generation, I really like everything Zeit does, but maybe a bit heavy for what we do. Also, supporting its routing system would be a gosh-darn nightmare, speaking from an ig-router perspective.
• Gatsby
  • If this wasn't so unbelievably heavy I might consider it an option: the GraphQL bit is really cool and would allow pretty effortless interfacing with AutographQL. That said, I use it for my personal blog and I get pissed off at Gatsby for introducing massive breaking changes every version...
• Eleventy
  • It's cool and nice and simple but it suffers from the same issue that Gatsby and Next do, which is that it's really not designed to host literally a single page. I admittedly haven't looked into it much.

Let's talk about this next week when I'm back in the office. It's a bigger piece of infrastructure work but we should totally do it if we can pull it off in a reasonable amount of time. 👍 👏 👏 🙌 🙌

* We didn't at the time because we thought SK's requirements too complex and not enough buy-in on React from the rest of Technology. Things have evolved substantially and I no longer think this is the case.

[aendra-rininsland]

Also, having just gotten back from a conference about doing all of this — let's think even bigger. What if we used something like NextJS or even Gatsby in conjunction with something like Netlify CMS? We could make interactive articles part of one single web-app. Updating components in one place updates every page to the latest styles. We don't have to worry about breaking old content during style/Origami updates because we have a single idiomatic model for building content. People ultimately author components and Markdown. We stop creating literal microsites for (pardon the profanity) every goddamn article.

It's an idea, is all I'm sayin'...

[maxharlow]

I've not spent long looking at Create React App et al, but my understanding of these tools is they're just collections of some rational default build options to get your project started, which you could then customise later. Given we'd want/need to do some customisations of that default config, adding FT-specific stuff, what does using CRA (or the others) give us?

One of the best things about our current setup is the isolated static nature of our deploys. Yes, there's duplication, but disc is cheap. And deploying like this has a couple of nice properties. One, when working on one project the risk of breaking every other project we've ever done is close to nil, no matter what you do. And two, the ongoing maintenance overhead for maintaining the S3 Bucket etc is almost nothing, as is the surface area we need to keep secure. So by all means let's explore Netlify CMS etc, but let's not give up these properties without good reason.

We should also consider doing little or nothing. We create perhaps a dozen IG pages per year, so let's be mindful of how long it would take for the time and effort spent on improving the process to be amortised across future projects.

[aendra-rininsland]

Closing as woefully outdated