mkavulich
commented
12 months ago I have done a lot of investigation on the best path forward for this issue, and I propose that it does not make sense to implement this until capgen features are merged into the main branch. The main reason I think this is that, currently, the "track variables" script must be run at the same directory level as the prebuild script, and limitations of the prebuild logic mean that this is actually above the top-level directory for this repository for actual use cases. I could envision setting up a test config file that would not be affected by this limitation, but this would not be reflective of actual real-world use cases for this script.
Additionally, the logic for this script will likely have to be reworked, at least slightly, for the new capgen way of doing things. This is a good thing, believe me, because the current script is a bit hacky. For example, one should be able to run this script from anywhere, not just a set directory, and it really shouldn't require specifying both a config file and a location for metadata files. But I don't think it makes sense to expend the effort to introduce a testing system with this refactor looming in the near-future.
Finally, I presume that once the capgen branch is merged to main the testing process will be able to be much cleaner: taking advantage of the newer code that (presumably) is much more flexible and object-oriented will allow for better CI testing. A script that does not require a prebuild config file could be tested by simply pointing to the test metadata and example suite files in the test/capgen_test directory.
Description
Implement ccpp_track_variables.py as a continuous integration test via GitHub actions.