Create integration tests for supported template formats

[shawnbot]

The simplest thing here would probably be to have a script that runs a standard set of converted templates (the feature lower common denominator) through a reference implementation of the target engine (ideally a shell one-liner) to ensure that we get the expected output. For instance, we might test the Handlebars conversion with a this, which we might call handlebars:

#!/usr/bin/env node
const fs = require('fs');
const op = require('yargs').argv;
const template = fs.readFileSync(op.template);
const data = JSON.parse(fs.readFileSync(op.data));
console.log(
  require('handlebars').compile(template)(data)
);

which the testing script could call with a diff.sh that looks something like:

#!/bin/bash
# transform the fixture template with the input format
meta-template fixtures/$(1)/template.njk -f $(2) \
  | ./bin/$(2) --data fixtures/$(1)/data.json \
  > fixtures/symbol/$(2).txt
# then diff it against the fixture's expected output
diff fixtures/$(1)/{output,$(2)}.txt

which you'd call with:

./bin/diff.sh symbol handlebars

(or do it in Node.) The layout might look like:

test/
├── fixtures
|   ├── lookup
|   |   ├── data.json
|   |   ├── template.njk
|   |   └── output.txt
|   └── symbol
|       ├── data.json
|       ├── template.njk
|       └── output.txt
├── bin
|   ├── diff.sh
|   ├── handlebars.sh
|   └── liquid.sh
├── handlebars.spec.js
└── liquid.spec.js

@dsingleton, do you have any thoughts on this?

[dsingleton]

👍 to the approach. I'm looking at something similar as a smoke test for components. I'll share the branch once i've got it up to date.

Running all the Fractal component variants through our transpilation process (which is meta-template plus some other bits) and asserting the output for the transpiled versions is the same for the original nunjucks.

Only question i've got is, would output.txt be commited? For my version i was thinking the reference output would be generated from the pre-meta-template nunjucks template. Where that would only change with breaking change in the nunjucks engine itself.

[shawnbot]

Thanks for the gut check. Some things I've reconsidered:

1. The engine-specific renderers in bin don't need to be sh/bash scripts; they can just nix the file extension and tell the shell what they run in the shebang, e.g. bin/nunjucks has #!/usr/bin/env node, and bin/erb gets #!/usr/bin/env ruby.
2. The name output.txt is ambiguous, and I would call these expected.txt and commit them.
3. The engine-specific outputs might not need to be written to disk if they can be diffed on the fly (maybe in Node, with some child_process.spawn() stdio magic).