As analysis design I want to use groupings with compound expressions

[drewcdisc]

Regarding using groupings with compound expressions.:

• [ ] Develop any model changes necessary.
• [ ] Create example that can test the model

[ASL-rmarshall]

While developing the FDA STF example, I tried to create some age group groupings using compound expressions. The compound expressions could be converted from the Excel template into YAML format, but conversion of YAML to JSON failed because the whereClauses in the CompoundGroupExpression class can currently only be specified as groupId references (not as fully specified conditions or where clauses). I've seen some LinkML examples using:

any_of:
  - inlined: true
  - inlined: false

(i.e. sub-classes can either be fully specified or referenced by identifier) but haven't managed to get this to work without getting an error at some point in the process (generation of YAML, conversion to JSON, etc.).

[ASL-rmarshall]

Additional thoughts:

• Individual where clauses will be easier to process/use if all components (i.e., the single condition or all subclauses in a compound expression) relate to a single dataset. This implies that dataset could/should be an attribute of an identified where clause (i.e., an attribute of AnalysisSet, DataSubset, Group, AnalysisGroup, and DataGroup instead of WhereClauseCondition): "this is a restriction that applies to the named dataset's variable(s)".
• If it's necessary to combine conditions for different datasets (e.g., in a data subset for an analysis that requires restrictions at both the subject level (ADSL) and the data record level in another dataset), then this could be achieved using a compound expression that references where clauses by identifier - each of which would have the dataset associated with the identifier (e.g., using a new, non-inlined attribute of something like "whereClauseId"). In this situation, it might also be helpful to define "join key" variables as metadata (to define how the datasets are joined), though this raises some questions:
  • How would you define join keys in an analysis that combines information for two datasets (or more, if that's possible), but whose definition does not include restrictions for all referenced datasets in some sort of where clause? Is more thought needed about defining the dataset and variable at the analysis level?
  • Is there any situation in which different groups within the same grouping factor might reference different datasets?
• It might be worth considering having two specializations of the WhereClauseCompoundExpression class:
  • One that has logicalOperator and inlined whereClauses (the current definition)
  • One that has logicalOperator, non-inlined whereClauseIds, and the ability to define join keys (but would need to figure out how to define these and it might be better to define join keys at a higher level).
• But this might get overly complicated with all the existing specializations of the WhereClauseCompoundExpression class (CompoundSetExpression, CompoundGroupExpression, CompoundSubsetExpression) - would you need two "flavours" of each...?

[ASL-rmarshall]

The model has been updated to support the definition of compound expressions using, in the whereClauses attribute, either of the following:

• A fully-defined WhereClause (containing either a condition or another compoundExpression):

  whereClauses:
    - level: ...
      order: ...
      condition:
        dataset: ...
        variable: ...
        comparator: ...
        value:
          - ...
    - level: ...
      order: ...
      compoundExpression:
        whereClauses:
          - ...

• A subClauseId attribute referencing another "identified WhereClause" by identifier value:

  whereClauses:
    - level: ...
      order: ...
      subClauseId: ...

This change has been applied for all types of WhereClause, including WhereClause, AnalysisSet, DataSubset and Group.