search

Mbed-TLS / mbedtls-framework

TF-PSA-Crypto and Mbed TLS version-independent build and test framework
Other
2 stars 16 forks source link

Move misc. "easy" scripts to the framework #87

Open mpg opened 7 hours ago

mpg commented 7 hours ago

Move the following files to the framework:

tests/scripts/psa_collect_statuses.py
tests/scripts/run-metatests.sh
tests/scripts/run_demos.py
tests/scripts/test_zeroize.gdb

These are the scripts that are:

To be discussed: should we also move programs/test/zeroize.c and programs/test/metatest.c?

Note: as usual for this EPIC, the work is only to move the files to the framework repo and adapt the paths everywhere they're used in mbedtls 3.6 and development. If further changes are needed to adapt to tf-psa-crypto, they're out of scope and will be covered by separate tasks.

gilles-peskine-arm commented 6 hours ago

tests/scripts/psa_collect_statuses.py

I wrote that a while ago to solve a short-term problem (coverage of error cases and comparing that with what the specification draft says, to identify missing cases in the specification and missing coverage in our tests). I got it merged because it seemed like it could be a useful way to measure this aspect of test coverage, but I don't think we've made any use of it since then. So we could remove it and the corresponding code, if that's easier than moving it to the framework.

Today I would use the PSA wrapper generator framework for this, and I even have code to do that (also logging parameters, not just return values) in https://github.com/Mbed-TLS/mbedtls/pull/8559 (I used that partly to validate the PSA wrapper generator framework and partly because I needed to log the coverage of parameters).

programs/test/zeroize.c

It's very tightly coupled with test_zeroize.gdb so I think both should be in the same repository. And I can't think of a reason why they should be different in different branches, so it makes sense for both to be in the framework.

programs/test/metatest.c?

In principle this could have branch-specific code. But right now I can't think of anything branch-specific that I'd want to add even if I had plenty of spare time. It's mostly about checking that we're catching the kinds of failure we expect to catch from the test framework as well as from environment features such as sanitizers. So I think it belongs together with tests/src, in the framework.

mpg commented 6 hours ago

My main question about programs is are we ready to have C executable programs (as opposed to libraries / object files) in the framework, in terms of how our build system(s) work? Cc @ronald-cron-arm

Other than that, I'm also inclined to have them in the framework. But if that's not easy, we might want to post-pone it (or at least split it to a separate task).

ronald-cron-arm commented 5 hours ago

My main question about programs is are we ready to have C executable programs (as opposed to libraries / object files) in the framework, in terms of how our build system(s) work? Cc @ronald-cron-arm

We have not done this so far thus we definitely need a specific task for that.