Add automated testing framework

khale commented 5 years ago

This PR adds a testing framework to Nautilus. This makes it much easier to test code you've written for some component. The meat of this PR is in 5d23e52. Key features include:

tests can be added within the component you're working on. No need to edit core kernel code. You can see how to use this in the contrib doc (modified in d7ac2c4) and in src/test/test.c.
tests that are added are automatically tied to the kernel command-line processing framework, so can be invoked easily by adding a kernel command-line flag (-test <testname> "arg1 arg2 arg3")
tests can be integrated with the automated build system (0e19246). This is done using a YAML-formatted test matrix (a4bf0c5). Note that each of these tests can be run for any number of configs. Two default configs are provided in this PR (see 10573b3).

Other changes:

adc7ce9 allows us to pass in a specific exit code to QEMU when it shuts down. Needed for scripted tests.
b16fd37 autogenerates an iso directory for each make isoimage build. This just cleans up some cruft in the source directory
e61279a makes build output for make isoimage less cluttered. Note: integration with the shell will be trivial once the shell rewrite PR is merged in.
d7ac2c4 adds supporting documentation for the testing framework

khale commented 5 years ago

This pull request introduces 6 alerts when merging d7ac2c43d7baee2c54847086c2ddc919134fbea7 into 29bf3429c302edc04fd57ab0ef7f4f745295dbd1 - view on LGTM.com

new alerts:

4 for Unused local variable
1 for Unused import
1 for Module is imported with 'import' and 'import from'

Comment posted by LGTM.com

khale commented 5 years ago

This pull request introduces 6 alerts when merging 5ae719f23b5213473906263ab8178e1de4d1e727 into 29bf3429c302edc04fd57ab0ef7f4f745295dbd1 - view on LGTM.com

new alerts:

4 for Unused local variable
1 for Unused import
1 for Module is imported with 'import' and 'import from'

Comment posted by LGTM.com

khale commented 5 years ago

This pull request introduces 6 alerts when merging e78623c896710797f14ede4ed3e7e2d1323a7bf3 into 29bf3429c302edc04fd57ab0ef7f4f745295dbd1 - view on LGTM.com

new alerts:

4 for Unused local variable
1 for Unused import
1 for Module is imported with 'import' and 'import from'

Comment posted by LGTM.com

khale commented 5 years ago

@PeterDinda thanks for the feedback.

existing tests are not incorporated, although I will be incorporating some that I'm in charge of (specifically Nemo related)
did not consider the child thread issue, will consider autoexec, or potentially the constraint that all tests are run in a newly spawned thread
the shell invocation for tests is my next planned addition for this---was waiting on the shell rewrite to go in first
no timeouts, but that's a good idea---presumably easier now with the revamped timer framework
other linker/init integration is also in the works, but is on hold until deadline season passes

khale commented 5 years ago

This pull request introduces 6 alerts when merging e6d9b191b1b2d251e238c588b12f8146950e0878 into 189083de8cf5cea7eabdf61ae8a472ab3316503c - view on LGTM.com

new alerts:

4 for Unused local variable
1 for Unused import
1 for Module is imported with 'import' and 'import from'

Comment posted by LGTM.com

khale commented 5 years ago

This pull request introduces 6 alerts when merging f2c7ce092f3b0e6dbd640056feb5b9ee95679da9 into 189083de8cf5cea7eabdf61ae8a472ab3316503c - view on LGTM.com

new alerts:

4 for Unused local variable
1 for Unused import
1 for Module is imported with 'import' and 'import from'

Comment posted by LGTM.com

HExSA-Lab / nautilus

Add automated testing framework #10