Support for Checked-C Macro Test cases

[dtarditi]

@arunkumarbhattar I see that these changes clone most of the existing Checked C tests and provide variants of them for the new syntax. I don't think it is a good idea to do this to provide tests for your changes. It'll slow future development on the project down. In general, if you have k features in software, there k! possible combinations of features. If you have n scenarios to test, writing O(k!) tests for those scenarios is too many tests. This testing problem has been studied in other situations. What you want to do is to choose different pairs of features to use so that you can cover the overall space. So you write m tests that pairwise cover the space.

Concretely, what does this mean? We need to consider where we expect to land.

• For Checked C, I recommend we move away from the : syntax that we've been using to separate Checked C syntax like bounds and interface types from existing C syntax. We already are introducing new keywords that can be used in a macroized way such as _Bounds, which render the : unecessary. It is too much to make this change in one step - there are additional tests and examples that need to be changed.
• I am also uncertain about the use of type qualifiers to specify the different types of checked pointer. After reading all the tests, I think the qualifiers are significantly harder to read. That means the code is less clear, which is not good when trying to write secure code! I think we'll want to experiment with other mechanisms: perhaps we should add a syntactic form _Retype(T) or _Refine(T) that allows a programmer to specify a more precise checked type for a declarator (a refinement type). That would require programmers typing more, but at least the code would be easier to use.

I recommend instead of cloning the files, you modify some of the tests in existing files to use the new syntax. You will still need new tests for some of the features, of course. I recommend:

• Sometimes use _Bounds instead of : bounds(. Do the same for _Count. When we eliminate the use of :, we can switch all the tests over. For now, we need to keep both syntaxes working.
• Sometime use the type qualifier syntax in place of the current _Ptr\_ArrayPtr syntax.

Given the syntactic nature of the changes, I think it is fine to only modify a small fraction of the tests.

An alternative is to macro-ize some tests so that they use either form. Than we can compile and run more tests, using macros to compile slightly different code. That is more work, though, and I don't recommend doing it to support syntactic forms that we will deprecate.

[arunkumarbhattar]

Thank you for the detailed explanation of your concerns regarding the current testing strategy.

Your recommendation of modifying existing tests to use the new syntax, instead of creating separate tests for each syntax variant, is a more efficient approach. This will ensure that both old and new syntaxes are tested without increasing the number of tests exponentially.

Consequently, I shall add the macro tests within the existing test suites at appropriate places.