Reduce overhead of Jest tests

[kirlat]

Right now, on order to test our code with Jest, we, except creating test suites, have to do two other things, namely:

• Produce a second node build that is necessary to run our code in Jest.
• Transpile code with Babel to make it compatible with Jest and the way it loads files (i.e. using require). Both those steps takes time. The first step increases overall build time, every time, even if we're not going to run Jest tests just yet. Also, we're submitting the node versions of files to GitHub and I think it is an unnecessary waste of space.

It would be great if we could find a way to eliminate the two steps listed above completely, or, if not possible, reduce the overhead they create.

Once ESM support landed in node.js, Jest started its work to support ES modules within its test environment. This work is probably not fully completed yet, but its results might be able to allow some improvements.

I think we can research if we can do the following to simplify testing:

• Try to load modules of the code currently being tested to Jest directly, without transpilation. This will allow it to eliminate Babel.
• Try to replace things that are not supported in Jest environment with polyfills. This might eliminate the need for special node builds.
• Try to create library bundles (data-models, inflection-tables, etc.) in an ESM format. This will have other benefits such as code becoming tree-shakable (will allow to eliminate classes that are not used).

Webpack does not support outputting of ESM jest yet, but it's getting there. But maybe for simple, "pure" library packages with little external dependencies and no special loader and transformer needs (such as data-models and client-adapters that do not use Vue.js, Sass, and other things alike) we can use Rollup temporarily. It should not be hard. Then, when Webpack will be ready, we can switch back to using it.

What do you think about those suggestions? Do they make sense? Would we benefit from the changes described above?

[balmas]

Some questions:

1) do we only use Babel now for Jest? Is the transpiling still necessary for browser support? 2) would it be possible to keep the node build as an option? It has been nice to be able to create command line tools that use the Alpheios code (e.g. see https://github.com/alpheios-project/lexical-tests-node). While that hasn't been updated in some time, I and others do still use it from time to time, and I don't want to rule out the possibility of updating it or creating some other tools like it. 3) How would switching to rollup for some libraries not others work with lerna and our new monorepo approach?

[kirlat]

1. do we only use Babel now for Jest? Is the transpiling still necessary for browser support?

Yes, it is for Jest only. We do not use any features that modern browsers do not support and thus. we don't need transpilers to build our distributable bundles. Jet, on the other hand, runs Babel before starting the tests if there are any new changes in the code. The following lines configure Jest's use of transpilers:

"transform": {
      "^.+\\.jsx?$": "babel-jest",
      ".*\\.(vue)$": "vue-jest",
      ".*\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$": "<rootDir>/fileTransform.js"
    }

2. would it be possible to keep the node build as an option?

Definitely! My biggest concern right now is that every time we do a dev build, we build our bundles twice: once for the browser version, and second time to produce a node build. That will add up a lot of build time when several small changes need to be made often. I also think there is probably no reason to keep the node version on GitHub because it's mostly a duplicate of a browser bundle.

But nothing prevents us from keeping an npm script that will build a node version when needed, maybe even called by the client packages (the ones that are going to use a node.js version).

3. How would switching to rollup for some libraries not others work with lerna and our new monorepo approach?

I think it should affect nothing, because in Lerna environment we're working with npm scripts. So as long as we'll replace a webpack's npm script with the analogous Rollup's one, nothing will change for Lerna.

Switching to Rollup is a decision I'm least sure about. It's is a temporary solution only, and we'll switch back to webpack once it will be ready to produce ESM output (I've read that many people will do that too).

But if switching to Rollup will be simple, why not to use it for now? If it would cause any troubles, I think we should leave things as they are now, with webpack, and not waste our time with Rollup configuration. What do you think?

[balmas]

But if switching to Rollup will be simple, why not to use it for now? If it would cause any troubles, I think we should leave things as they are now, with webpack, and not waste our time with Rollup configuration. What do you think?

I would prefer not to switch to Rollup if we don't have to, just because I think the build is already complicated enough.

[kirlat]

I've tested a loading of ESM modules in Jest without transpiling it first and it works, generally. However, there are issues with some imported packages, uuid, namely. The problem is that Jest dependency loader does not support an "exports" field in package.json yet. It leads to problems described in this StackOverflow question.

Since removing a transpilations step from Jest testing would be nice, but is not critical right now, maybe we can wait until this will be resolved on the Jest side? We can track progress in https://github.com/facebook/jest/issues/9771. What do you think?

[balmas]

ok