Per a discussion between @iay and @philsmart today, the long term direction is that test data is ultimately going to be passed through every ruleset used by the production tooling policies. This means we need to be more careful about the test data we use; in general it needs to be valid for rulesets we haven't even added to the testbed yet and the best way to achieve this is by making it the smallest possible variation from fully correct metadata possible.
One specific change we can make which will reduce the number of future issues of this kind is to add checking for schema validity to the default validator, in the way we do this in the production tooling.
We can and probably should treat this as an additional ruleset we're testing, and have at least one unit test for it in the usual way. A good one might be to just not have any role descriptors (one is required, unless you have an AffiliationDescriptor). But we don't need to go much further; this is mainly a safety check.
This will require some of the existing test data to be revised.
philsmart
commented
1 year ago
Per a discussion between @iay and @philsmart today, the long term direction is that test data is ultimately going to be passed through every ruleset used by the production tooling policies. This means we need to be more careful about the test data we use; in general it needs to be valid for rulesets we haven't even added to the testbed yet and the best way to achieve this is by making it the smallest possible variation from fully correct metadata possible.
One specific change we can make which will reduce the number of future issues of this kind is to add checking for schema validity to the default validator, in the way we do this in the production tooling.
We can and probably should treat this as an additional ruleset we're testing, and have at least one unit test for it in the usual way. A good one might be to just not have any role descriptors (one is required, unless you have an
AffiliationDescriptor). But we don't need to go much further; this is mainly a safety check.This will require some of the existing test data to be revised.