jokiefer
commented
3 years ago UI
We propose to add a dropdown Menu/Button to the places, where a ConformityCheckRun should be triggered for a specific Metadata object. Such a dropdown lists all ConformityCheckConfigurations that are allowed for the given metadata_type.
Does that match your expectations?
response_map
As we store the complete response of a single run in the response field as JSON, I doubt there is the need for the errors and additional_information fields. We are able to generate an HTML representation of the response object that can look like this:
Is there any additional use for the fields error and additional_information, besides giving feedback to users?
If not, we propose following parameter_map object:
{
"status_identifier": "EtfItemCollection.testRuns.TestRun.status",
"error_values": ["FAILED"],
"success_values": ["PASSED", "WARNING"]
}Here, success_identifier contains the path to the key that holds the information about the status of the CheckRun. The fields error_values and success_values hold the values of the success_identifier that determine a failed or a passed CheckRun, respectively.
Example:
A given response
{
"EtfItemCollection": [
{
"testRuns": [
{
"TestRun": {
"status": "PASSED"
}
}
]
}
]
}would be evalutated as successful test, whereas
{
"EtfItemCollection": [
{
"testRuns": [
{
"TestRun": {
"status": "FAILED"
}
}
]
}
]
}would be evaluated as failed test. In this scenario, a result with "status": "WARNING" would also be evaluated as successful test.
Please let us know, if the proposed changes match your expectations.
jansule
Requirements - outdated!
qualityRulename- CharFieldfield_name- CharField (with choices)property- CharField (with choices)operator- CharField (with choices)threshold- IntegerFieldRuleSetname- CharFieldrules- M2MField (onRule)ConformityCheckConfigurationname- CharFieldConformityCheckConfigurationExternalinherits fromConformityCheckConfigurationexternal_url- URLFieldparameter_map- TextField (contains json)response_map- TextField (contains json)ConformityCheckConfigurationInternalinherits fromConformityCheckConfigurationmandatoryrulesets- M2MField (onRuleSet)optionalrulesets- M2MField (onRuleSet)ConformityCheckRunmetadata- ForeignKey (onMetadata)confirmity_check_configuration- ForeignKey (onConformityCheckConfiguration)time_start- DateTimeFieldtime_stop- DateTimeFielderrors- TextField (contains json)additional_info- TextField (contains json)Requirements reduced 2020-10-15
New app must be created:
qualityNew models must be created
ConformityCheckConfigurationname- CharFieldmetadata_types- TextField (contains json) - metadata.metadata_type, view formatConformityCheckConfigurationExternalinherits fromConformityCheckConfigurationexternal_url- URLField - APIapi_configuration- TextField (contains json) - defines test classes, result parsing, ...ConformityCheckRun- async - see pending tasks - typevalidatemetadata- ForeignKey (onMetadata)conformity_check_configuration- ForeignKey (onConformityCheckConfiguration)time_start- DateTimeFieldtime_stop- DateTimeFielderrors- TextField (contains json)passed- Booleanresult- TextField (contains json, xml, html)Conformity checks
We need to separate between internal and external confirmity checks. Please regard the attached diagram to get an idea of the structure.
ConformityCheckConfiguration
Base model for
ConformityCheckConfigurationExternaland...Internal.name
The name of the configuration
ConformityCheckConfigurationInternal
Holds the configuration for an internal conformity check.
mandatory_rule_sets
A set of ruleSet records, which are mandatory to pass successfully.
optional_rule_sets
A set of ruleSet records, which are nice if passed successfully.
ConformityCheckConfigurationExternal
Holds the configuration for an external conformity check.
external_url
A link pointing to an external test API.
parameter_map
Holds json as text. Further details below.
response_map
Holds json as text. Further details below.
ConformityCheckRun
Holds the relation of a metadata record to the results of a check
metadata
A ForeignKey to the related metadata
conformity_check_configuration
A ForeignKey to a ConformityCheckConfiguration record.
time_start
DateTime when the test started
time_stop
DateTime when the test ended
errors
Holds json as text.
additional_info
Holds json as text.
ConformityCheckInternal
Internal checks are based on
Rules andRuleSets.RuleSet
Groups rules and holds the results of a rule check run. RuleSets iterate over their rules and perform the rule checks.
name
The name of a ruleSet.
rules
A set of rules.
Rule
name
The name of a rule.
field_name
field_namedefines an attribute of the Metadata model, such asabstractorkeywords. A list of valid choices must be provided, so the rules can be easily created.Valid choices are
titleabstractaccess_constraintskeywordsformatsreference_systemproperty
A property describes what can be measured on the
field_name. E.g.abstractcan have the propertylen, since it's a string.keywordsis a queryset of records and can have the propertycount, which is semantically equal tolenbut reduces the query time.For now there are only these two properties which can be checked. The list of choices can be extended in the future.
operator
Defines a mathematical operator, which is used to describe the expected
field_nameproperty.Valid choices are:
>>=<<===!=How a rule could be used
A rule could be checked like this
ConformityCheckExternal
Mandatory attributes for this class are:
external_url
Points to an online API
parameter_map
Maps the metadata records field names on the required parameters for the API.
Example: We have a metadata object with a field named
get_capabilities_uri, which has to be used as the parameterinputURIof the API:Example implementation for usage
response_map
Maps the known response elements to a given structure. This way, we are aware of how, e.g. the differen error elements of the response are named, so we can parse the response in a generic way.
Example:
error_identifier
Every API must have at least one error element, which contains the found errors during the test. It may be, that there is not just one error element, but rather multiple error elements for different types of error (TypeError, OutOfRangeError, ...). All these error element identifiers have to be collected in a list, which we call
error_identifieradditional_info_identifier
Some APIs might provide warnings or hints as well, which are not really errors but some kind of additional information. We proceed here the same as we did for
error_identifiers: Put all these hints or warning element identifier in a list, which is calledadditional_info_identifier.success_identifier
Every test API should have one element in response, which states whether the test run was successful or failed for the input. This element has to be named by using
success_identifierExample implementation for usage
Please note: The function
get_val_for_identifier(id: str)does not exist yet and I didn't give any example of how to implement it. This function would be more complex, since it has to handle json, as well as xml responses, in case we deal with a test API, that uses xml.