not_contains_all not working correctly with get_column_order_from_dataset

[gerrycampion]

Links to related JIRA Tickets

• https://jira.cdisc.org/browse/CORERULES-7

Rule Information

• Standard: SDTMIG
• Rule ID: CG0014 (A)
• Rule Description: Variable: GEN Condition: Variable Core Status = Required Rule: Variable present in dataset ( and ^= null ) <-- Not included in Part A

Describe the bug not_contains_all operator not working correctly with get_column_order_from_dataset operation. Refer to CG0014 (A) in the rule editor and corresponding negative test data.

Rule Type: Dataset Metadata Check
Operations:
  - id: $required_variables
    operator: required_variables
  - id: $dataset_variables
    operator: get_column_order_from_dataset
Check:
  all:
    - name: $dataset_variables
      operator: not_contains_all
      value:
        - $required_variables

get_column_order_from_dataset appears to return a list object, but not_contains_all expects self.value[target] to be a series object in order to apply unique().

Error returned from Rule Engine

{
  "error": "An unknown exception has occurred",
  "message": "unhashable type: 'list'"
}

Expected behavior No errors. The rule should return no issues when all required variables exist or return an issue for missing required variables.

[nhaydel]

@gerrycampion why not just use variable_name as the target instead with a rule type of Variable Metadata Check:

i.e:

Check:
  all:
    - name: variable_name
      operator: not_contains_all
      value:
        - $required_variables

I get that we may want to fix this, but is the rule actually blocked by this bug?

[gerrycampion]

No, that should work, thanks. I updated the issue title

[ab3263266]

Can I ask variable_name gives the list of all the variables in the dataset? If yes it is very unclear

[gerrycampion]

Can I ask variable_name gives the list of all the variables in the dataset? If yes it is very unclear

variable_name is one of the metadata variables available when using the Variable Metadata Check rule type.

The Variable Metadata Check rule type produces a data frame containing one record for each variable in the current dataset. At the dataset level, you are correct that variable_name gives the list of all variables in the dataset. This means that at the record level, variable_name gives the name of the variable of the current record.

@ab3263266