It is quite difficult to run auto generated tests using igortest. A current approach with the existing igortest could be:
Perform the first test run. (must be inside an IM)
Each test case can generate new code (as a string) and save it as a new file in a specified path.
Run compilation tests using the generated code.
Perform the second test run.
Load all code from the specified path.
Search for test cases there and execute them.
Cleanup all auto generated code.
This results in two test runs and multiple coverage and test result files. You also need to change your CI code to combine all results. It is also difficult to associate the generated code with the generator function.
A simpler approach could be to create new assertion/helper functions that:
Take the new code as a string and save them at a temporary path known to igortest.
Trigger a compile check (like our compilation assertions). The generated code can also intentionally fail!
Only if the generated code compiles and is intended to be compiled:
Search for all new test cases in the generated code and inject them after the current test case in our test schedule. The test case names of the injected ones are all prefixed with the current test case name. They all share the same test suite!
Execute the new test cases normally.
Cleanup all generated code (if desired they can be kept for debugging purpose)
Continue with the usual test schedule
Auto generated code can be quite useful if you need lots of combinations that is difficult to test with usual code and needs to be written one by one. It is also quite useful if you want lots of compilation checks with small snippets of code.
It is quite difficult to run auto generated tests using igortest. A current approach with the existing igortest could be:
This results in two test runs and multiple coverage and test result files. You also need to change your CI code to combine all results. It is also difficult to associate the generated code with the generator function.
A simpler approach could be to create new assertion/helper functions that:
Auto generated code can be quite useful if you need lots of combinations that is difficult to test with usual code and needs to be written one by one. It is also quite useful if you want lots of compilation checks with small snippets of code.