Share test harness with `isomorphic-git`?

[billiegoose]

@jvilk I've got a problem in isomorphic-git that you've solved for BrowserFS, which is how to load filesystem fixtures and run unit tests in the browser.

Believe it or not, despite making the backend test thing for a previous PR, I don't understand how the test harness in BrowserFS works. It looks like it involves browserify and rollup and webpack and TypeScript? I know this is a bit of a stretch, but I wondered if you'd be interested in explaining to me a bit more how it works, and in exchange I could help document/refactor the test harness into a shared npm package that both BrowserFS and isomorphic-git could use. Thoughts?

[jvilk]

Sure, I can explain things a bit, although I don't know if any of it is general purpose or reusable. It's convoluted. The unit tests do not run against a bundled version of BrowserFS; instead, it directly bundles the test and source modules into a single JavaScript file. I did this for two reasons: 1) convenience, and 2) because folks will consume BFS in the same way from NPM.

Here's how the build process proceeds:

• TypeScript compiles the script directory into build/scripts.
• scripts/make_test_launcher.ts generates test/harness/run.ts from test/harness/run.tstemplate
• test/harness/make_fixtures_loader.ts generates test/fixtures/load_fixtures.ts.
  • This file contains all of the fixture files as base64 strings, which is the easiest way to encode binary data within JavaScript source files. Also, the Node API natively supports base64 as an encoding option.
  • The file first creates all of the needed fixture directories (mcb), then it writes all of the files into the file system (fcb).
  • This file generates TypeScript so it's possible to consume it as a module from TypeScript. For all writable file systems, the test code runs the method exposed by this module to populate the file system with filesystem fixtures.
• TypeScript compiles src/test directories into build/temp/tests/ts.
• Rollup bundles build/temp/tests/ts/test/harness/run.js into build/temp/tests/rollup/test.rollup.js
  • (It also bundles up a separate bundle for the WebWorker we use to test WorkerFS.)
• Finally, Webpack bundles build/temp/tests/rollup/test.rollup.js (& the worker rollup bundle) into test/harness/test.js.

Like with our regular bundles, Webpack is needed to support global Buffer references and some Node polyfills. I have not revisited this decision since to see if Rollup works sufficiently well to no longer use Webpack (or visa-versa). (I use Rollup + Webpack because Rollup significantly reduced bundle sizes. I have no idea how well Webpack 2+ works in this respect.)

Hope that helps, and hope I'm not too late with this information!

[billiegoose]

You may be right... it may not be as general purpose/reusable as I had hoped. I like the Base64 encoding idea though. I might be able to use that somehow....

What I ended up creating for isomorphic-git's tests is a, well, isomorphic fixture factory. The crux of it is:

async function makeFixture (name) {
  return process.browser ? makeBrowserFixture(name) : makeNodeFixture(name)
}

where makeFixture returns a { fs, dir } each time it's called. makeNodeFixture creates a temporary directory, copies the fixture files in name into the temp dir, and returns { fs, dir = tempdir }. makeBrowserFixture returns an OverlayFS of a readable HTTPRequestFS that hosts all the fixture directories and a writeable MemoryFS, and each time makeBrowserFixture is called it empties the MemoryFS to reset any writes made by an earlier test.

It ends up being two completely different mechanisms but manages to provide the same API for setting up fixtures. But since it uses BrowserFS to provide the fixture, that strategy probably can't be used in BrowserFS's own unit tests. ¯\_(ツ)_/¯