Handle unit tests

[rylev]

Background

Currently spin-test only handles integration style testing where the component's trigger export is invoked and the app is run from start to finish. However, it would also be useful to handle unit test style tests where sub-functionality private to the Spin application can be tested.

Currently, it is possible in most toolchains to write regular unit tests and run them as native code using the language's test runner of choice (e.g., writing #[test] annotated test functions in Rust and running them as native code with cargo test).

The issue

The issue however is that functionality might be slightly different between the native target and the WebAssembly targeting meaning users can not be 100% sure that their unit tests are testing the code that will be run when running as WebAssembly. This is not only true of languages like Rust which use compile time flags (i.e., cfg) to switch code out depending on which target, but also dynamic languages like JavaScript which might use an entirely different execution environment than when running outside of WebAssembly (e.g., node vs StarlingMonkey).

What's more, users may wish to blur the lines between unit tests and integration tests and have some unit tests that test internal functionality and still do some sort of I/O. For example, the spin-fileserver project has some unit tests that read from fermyon:spin/config. This is not possible to test today using native test runners.

Possible solutions

One possible solution would be to provide some mechanism that allows spin-test to target the Spin app's component directly as a source of spin-test compliant tests (instead of having the test components come from some other component).

More than likely we would want spin-test to have a different code path for this unit test style approach as treating the app itself as a test component comes with lots of complications caused by composition. I can imagine spin-test always checking whether the Spin app's component exposes spin-test-* or run exports and then running those if it finds any.

[fibonacci1729]

This makes total sense. I think the possible solution you suggest is worthwhile to explore. Could you say more about the complications caused by composition you are anticipating?

[rylev]

What I meant by "complications caused by composition" is that the current code path expects the test component to have a certain shape that the app component is unlikely to have. For example, we expect the test component to import the http-helper interface which an app component will not do. In short, treating an app component as a test component is hard, and it's likely that just having a separate code path that assumes we're running an app with unit tests, might be easier to implement.