Initial test scaffolding and rudimentary tests

mstade commented 8 years ago

Description

This PR adds some test scaffolding in test/fixture.bash, as well as some initial tests. These are for testing the CLI functionality, so the tests are written in bash and run with bats.

Motivation and Context

We want test because we're not quite as cowboy as we'd like to think. Also see #4.

How Was This Tested?

There are two kinds of tests:

CLI tests, implemented in bash using bats
Unit tests, implemented in javascript, using tape

The idea behind using bash for CLI tests is that it's closer to the actual usage of ez-build that it'd be if setting up the tests using node child process APIs. Ideally, these tests should probably be run in a container, to ensure a consistent test environment, but that'll have to be future work if need be. (Wercker supports this by using Docker.)

The combination of CLI and unit tests should hopefully enable a more robust test suite, since we're effectively testing individual units as well as the integration by testing the final CLI.

(See below discussion for further details to using bash instead of JS for CLI tests.)

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
Checklist:

[x] My change follows the style of this project
[ ] My change requires a change to the documentation
[ ] I have updated the documentation accordingly
[x] I have read the contribution guidelines
[x] I have added tests to cover my changes
[x] All new and existing tests passed

Poetro commented 8 years ago

I'm not sure that in a Node.js project it is a good idea to write tests in bash. Node has all the tools that are required to run apps and check output, and to read folder structures (even in synchronous ways, if that is required.)

mstade commented 8 years ago

Sure, my point with writing and running the tests in bash is that that's the environment under which the tool runs. I know full well that node can execute processes and do all of the file operations necessary, but it's cumbersome, doesn't really reflect the reality of how the cli is executed, and bash is eminently suited for the purpose of shell scripting.

We'll probably need tests running under node as well to test individual units, and I'm working on that, but I do think there's plenty of value in using bash to test the cli.

mstade commented 8 years ago

@Poetro I've updated the PR description with a bit more detailed rationale on why I decided to go with bash + node, a opposed to just node. Hopefully that makes sense!

Poetro commented 8 years ago

:+1:

mstade commented 8 years ago

Thanks @Poetro, I'll release and merge!

zambezi / ez-build