test: migrate from aegir to polendina

[mikeal]

This PR moves from aegir to smaller tools that accomplish everything we do with aegir. Namely, this directly uses standard, mocha and polendina (for browser testing). This drastically reduces npm install time and the tests run much faster as well.

[mikeal]

Aegir is not only about testing, but also about code linting and doing releases. You only need to upgrade Aegir and not all dependencies separately, that lowers the maintenance burden. Also not using it would split the ipfs-related ecosystem.

We don’t actually use anything unique to aegir.

• The rules other than just what is in standard for linting are quite minimal and don’t really justify using a less familiar linting ruleset. To me, a few additional linter rules just aren’t worth adding a barrier to entry for new contributors. We may also get a much faster lint since we don’t have rules that require babel during linting.

• We also don’t use any of the build tools in aegir, we’re just doing a regular npm test and publish phase automated with GitHub Actions.

• That you “only have to upgrade aegir” is also the source of the problem. Aegir includes everything anyone in a related project might need, which is a huge set of things. Since we only use a minimal set this includes a lot of code we never use that slows down installs, increases the size of node_modules, and slows down the test runs. The way to make things faster is usually to simply do less.

• Periodically upgrading 3 devDependencies is not a significantly larger burden than upgrading 1 and the benefits are quite clear (roughly 1/3 the npm install time on my machine). In fact, since we have 3 other devDependencies this is really the difference between 4 and 6 devDeps to manage.

A few final notes:

• I’m skeptical of all these highly specific practices being embedded in a single tool. It makes altering our practices and adopting new or better tools very difficult. It also increases the distance between regular contributors and new contributors, since we’re using a toolchain and practices highly specific to us and not necessarily in alignment with what the rest of the community is using.

• As a general policy, we should try to avoid creating and standardizing on project specific tooling. If we can avoid this we also avoid creating a wider chasm between us and potential contributors. The “consistency” argument for using the same tooling everywhere is quite shallow when that tooling is only adopted by us and not by the rest of the community. In other words, we’re already using tooling inconsistent with the broader community so there’s very little consistency argument to be made.

• If there is a balanced tradeoff to be made between familiarity for full-time paid contributors and new contributors I’m always going to take the latter. I’m just not all that concerned with people that we pay full time doing a little more work so that we are more accessible to the millions of developers we don’t pay.

• I’m hesitant to adopt any project, from anybody, that is doing a dozen things. Sometimes it’s unavoidable because it’s the only way to do something, but when I have no other choice I tend to take a mental note that this is debt we’ll have to pay down later. I adopted aegir specifically to solve cross-platform (browser and node.js) testing, which it did quite well, and now that there’s something that accomplishes this faster with fewer deps it’s an obvious choice.

I would rather suggest that polendina gets integrated into Aegir.

That may happen, but it wouldn’t change how I would want to adopt it here. If all I’m getting out of aegir is the browser testing I don’t see any reason to include everything else it does. Aegir may have very good reasons for not adopting polendina that simply aren’t a concern for this module.

js-ipfs is a very large project with some very unique problems and those are driving the needs of aegir, as they should be. For aegir to be a great tool for js-ipfs it will become increasingly problematic for small modules like js-block and that’s an expected tradeoff we shouldn’t worry too much about. One of the benefits of small modules is that you don’t have to make big project problems everyone’s problem.

[mikeal]

GitHub Action Performance Comparison:

Note: since we are able to use the standard action image instead of the puppeteer docker image we skip a large phase in the beginning.

• New Branch

• Latest Release

[rvagg]

I agree with all of that ^.

The size and dependency bloat problem for me is the major one though. Something that has a few plain dependencies shouldn't take an age to install its devDependencies. Simple should be simple. This repo, after npm install takes up 500Mb of space, spread across many dependencies, not just one large one. This is a simple project, and that size is unreasonable. I have a large dev directory with many active projects and I can't afford all of them being that size. polendina, if you bypass the chrome install and do a global (info in the readme on that) takes up 32Mb. standard is a little extra. I can manage that. Still more than I'd like but it's mainly Webpack's fault.

When I encounter a project with a hugely complex dependency graph I won't contribute to it unless I'm absolutely desperate. The install time, disk space and additional mental overhead suggested by a large dependency tree is rarely worth it. I also use dependency count as a code quality indicator. If you're pulling in lodash to do something you could do with Array#reduce then it's probably code I don't want to trust too far. An npm install that keeps going for pages and pages sets off red flags like this for me. I know I'm not alone in using such heuristics in this massive ecosystem.

[mikeal]

Bonus!

Now that we are just using the default github actions container I don’t need to worry about the puppeteer image changing and altering the cache key so... I was able to add node_module caching and skip the entire build phase when you don’t change the modules in package.json!!!

This branch now brings down the CI time from 5-9 minutes (depending on container caching) to less than 2 minutes w/ an extra minute added if you alter package.json.

[mikeal]

Aegir e.g. also does pre-push hooks, commit lints, changelogs, updating the contributors (and probably more).

These are great features for js-ipfs but we aren’t using any of these features either.

• pre-push hooks - we don’t use the ones they setup and if we ever decide to add hooks it would be slightly different ones anyway.
• changelogs - we release every commit to master so changelogs aren’t very useful.
• updating contributors file - i never understood why this is a thing, you can get the list of contributors from the git metadata, there’s a tab on GitHub that shows all contributors and data about those contributions, the CONTRIBUTORS file seems like one of those things we are still keeping around as a practice we ported from the pre-git days of open source.