Custom validation checks: update specification

[kd-ods]

Existing custom tests (aka 'additional checks) are:

• entity_identifier_scheme_not_known
• inconsistent_schema_version_used
• wrong_address_type_used
• person_birth_year_too_early
• person_birth_year_too_late
• ownership_or_control_statement_has_is_compontent_and_component_statement_ids
• statement_is_beneficialOwnershipOrControl_but_no_person_specified
• alternative_address_with_no_other_address_types
• statement_is_component_but_is_after_use_in_component_statement_id
• entity_statement_out_of_order
• person_statement_out_of_order
• entity_statement_not_used_in_ownership_or_control_statement
• statement_is_component_but_not_used_in_component_statement_ids
• person_statement_not_used_in_ownership_or_control_statement
• entity_statement_missing
• person_statement_missing
• component_statement_id_not_in_package
• duplicate_statement_id

We need to set up some way of specifying (for now and into the future) which custom tests are needed for each version of the data standard. We can probably build on this sheet which was created to support the building and testing of the in-schema validation checks for 0.4.

Edit: 17-06-2024

The list above needs to be supplemented with the following, I think:

• has_public_listing_information_but_has_public_listing_is_false
• entity_security_listing_operating_market_identifier_code_set_but_not_market_identifier_code
• entity_security_listing_market_identifier_code_set_but_not_operating_market_identifier_code

(See the errors defined at the end of this script.)

[kd-ods]

I've made a new BODS validation specification sheet (based on info from the old one). I'm currently working through the following.

• [x] Review all schema files and make sure that constraints are accurately reflected in column C
• [x] Review all normative docs pages and add any non-schema testable constraints as rows
• [x] In column E, note which constraints should have a custom test (aka 'additional check') or invite statistical feedback.
• [x] Use the 'Error...' column to record where in lib-cove-bods relevant code sits
• [x] Ensure that required new custom tests and statistical feedback are flagged, with any supporting specifications

[kd-ods]

If we think that the BODS validation specification sheet is going to be useful enough to warrant maintenance as development of BODS continues then, also:

• [ ] Add to PR template in main data standard repo 'I've updated the BODS validation specification if necessary. See Handbook' and edit handbook.

[kd-ods]

The specification sheet is now ready to be used as a reference point for work on lib-cove-bods and maintaining checks for BODS constraints.

The errors currently coded in lib-cove-bods (as listed at the top of this thread) are all added next to one or multiple constraints in the sheet (apart from the exceptions listed below). When updating the code for these errors to handle BODS 0.4, please check the sheet for anything useful in the Notes column. There should also be a ticket per error (linked in the 'Related live ticket' column) where @kathryn-ods can share valid and invalid BODS 0.4 JSON to help with test-driven development.

These are the exceptional errors for which I can't see current constraints in BODS. This might be because they were only relevant in previous BODS versions.

• ownership_or_control_statement_has_is_compontent_and_component_statement_ids
• entity_statement_not_used_in_ownership_or_control_statement
• person_statement_not_used_in_ownership_or_control_statement

These final two are useful checks, even though what a valid BODS array contains is rather underspecified atm. So they should be maintained for BODS 0.4.

[kd-ods]

@radix0000 , @kathryn-ods - I'm happy for this ticket to be closed, but I'll leave it to you two to do it next week when you're both underway on the work.