search

testdouble / scripty

Because no one should be shell-scripting inside a JSON file.
MIT License
963 stars 23 forks source link

more info #15

Closed opcodewriter closed 8 years ago

opcodewriter commented 8 years ago

I'd like to read more how scripty can solve the problems of using npm as a build tool. Here's a good article I think which describes some common problems: https://medium.com/@leo/should-npm-really-be-used-as-a-built-tool-e8ff4f6cbcef Can scripty help with that? If yes, how? I'd love to read a blog post or something based on real experience. Thanks!

searls commented 8 years ago

Solutions

I don't want to bury the lede here, but the tl;dr is my long-term hope with scripty is to add support for plugin modules which provide a common set of scripts to its search paths. Imagine you're building a simple node lib module in ES5 & standard, maybe you could just install the scripty-pack-testdouble-es5 and get @testdouble's default set of testing & versioning scripts without redefining them in your own repo (but while still being able to override any of them locally or by editing your package.json).

My goal with this is to enable scripty to start providing a way to solve common sets of problems without adding a bunch of unmaintainable cruft to each and every repo, particularly for classes of problems other than "build a front-end web app".

For more on the background of my thinking, here are my notes:

Clarify the issue

I view the world of npm modules I work on in two very different categories:

  1. Builds front-end static assets with that massive category of concerns (multi-stage transpilation, front-end testing, asset fingerprint, CDN deploy & invalidation, etc.)
  2. Everything else

Regarding (1), My opinion is that there is already a great example for dealing with this, and it's @ember-cli. For those who don't know, the ember specific bits of CLI could at this point be defined as optional add-ons, leaving behind a merely excellent set of dynamic defaults for building what at this point is a pretty well-understood problem. Despite the Node community's aversion against large modules, frameworks, etc., the truth is that big, complex problems tend to require big, complex code. You can choose whether to reinvent and maintain all that complexity in your repo if you're not in a hurry, but it seems like a waste to me.

Regarding (2), the vast majority of other npm modules I work on are itty bitty by comparison, and their automation needs are almost all one-off, and not well-served by other build tools (aside from maybe make). To a very small module, the complexity of a repo's Gulp/Grunt/Karma configuration can quickly outweigh the module itself (which is why I wrote teenytest along with scripty to spare myself). The real risk of one-off Gulp/Grunt configurations isn't their huge node_modules/ installs, nor their "overhead" or "heaviness", it's that teams rarely understand what their build is doing, leading to myriad problems that either go unfixed or are merely papered over. Once the tangled mess of a configuration is identified as the problem, any attempts to detangle or replace it will inevitably reveal unexpected and problematic points of coupling between the code and the configuration of the build tool.

So, sure, to the extent npm scripts aren't particularly expressive is a feature in a world reacting to the massive surface area of other popular build tools right now. My goal with scripty is to give a gentle offramp to something that's still maintainable without adopting a much more complex tool just to break things out into smaller files.

Reactions to the blog post

  1. I like the part where he says I'm awesome. That was cool.
  2. npm scripts does not provide a path to concepts like project comprehension, asset filtering, and so on, which makes it a risky approach if what you're building is a web application with front-end assets that need to be compiled and delivered
    1. I am not a @gruntjs hater at all. We wrote and use @linemanjs on top of Grunt
    2. I actually love @broccolijs and I think @joliss should win a medal for the amount of care and attention she put into the design of the API -- (enough that I started building @cabbagejs before moving on to help @ember-cli)
    3. If you know your project needs to build static assets for the web, I recommend searching for a solution that outsources the problem instead of shoehorning an amateurish configuration into your own repo (which is why I started @linemanjs as a response to huge one-off Gruntfiles, and why I think @ember-cli's propogation of @broccolijs to other front-end communities is important)
  3. I'd take the author's messy example of npm scripts over any Gulpfile, Gruntfile, or Brocfile I've ever seen after it's been lived in for a while; I envy the author, because that wasn't nearly as damning an example as he seemed to suggest it was IME.
  4. My preferred "solution" to each of the author's stated problems is, for each well-worn, well-understood domain area (e.g. web apps with front-end asset compilation) is a convention-based, pluggable build system with a large set of dynamically-set default behavior and lifecycle stages (like @ember-cli). This will get the wheel reinvention out of each individual module, with appropriate hooks for plugin modules to adjust behavior to taste
opcodewriter commented 8 years ago

First of all, wow. It's the most comprehensive response I've personally ever got on GitHub.

"My goal with scripty is to give a gentle offramp to something that's still maintainable without adopting a much more complex tool just to break things out into smaller files."

I think that's the real challenge: create a powerful and versatile tool without reinventing existing ones (making same mistakes).