PRUNERS / FLiT

A project to quickly detect discrepancies in floating point computation across hardware, compilers, libraries and software.
Other
36 stars 6 forks source link

Install FLiT's automated tests #285

Open mikebentley15 opened 5 years ago

mikebentley15 commented 5 years ago

Feature Request

Describe the new feature: We install pretty much everything, except we do not install the tests that test FLiT itself. It may be beneficial to install at least the relevant tests (e.g., do not install the test the checks that we can install FLiT).

Currently, the only automated tests installed are those inside of the python scripts (as doctests). One reason to install the tests is so that the doctests within the python scripts can reuse some of the testing infrastructure used by the FLiT automated tests.

Suggested change:

This could make for more rigorous testing by running the automated tests after installation to make sure the installation did not break any functionality.

Alternative approaches: Just install the test_harness.py somewhere to be used by tests and add a variable in flitconfig.py to find it. We don't necessarily need the tests installed. I doubt anyone would even notice that they are even there nor know how to run them. But, if we install test_harness.py somewhere (e.g., $PREFIX/share/flit/test_harness/test_harness.py), then we can comfortably use them from within the doctests. I just worry about having broken doctests after installation.

mikebentley15 commented 5 years ago

Specifically, I want to simplify the doctests for flit_bisect.py function run_make().

mikebentley15 commented 5 years ago

Also, simplify doctests for extract_make_var() and extract_make_vars() in flitutil.py.