cdisc-org / cdisc-rules-engine

Open source offering of the cdisc rules engine
MIT License
49 stars 13 forks source link

Update `distinct` operation to work with DDF entities. #687

Open ASL-rmarshall opened 12 months ago

ASL-rmarshall commented 12 months ago

The distinct operation appears to expect domain and SDTM-like variable names. If this is the case, either it should be updated to handle entities and attribute names, or a similar, entity-specific operation should be created.

nickdedonder commented 6 months ago

Needs more information. Test rule and test data needed

ASL-rmarshall commented 6 months ago

Rule DDF00041 can be used to illustrate this issue - test data is available in the unitTesting SharePoint

In the Rules editor, values for the domain and group attributes of the distinct operation are highlighted as invalid: image

The rule will execute correctly in test mode in the Rules Editor - when the test data is read in from the Excel test data template, each sheet creates a dataset whose domain value is derived from the Excel sheet name (.xpt extension removed and converted to upper case). However, when the rule is executed via the CLI (on data read in from a USDM JSON file), domain is not assigned for the "datasets" created by the USDM data service, so the rule does not execute correctly.

This issue is likely to be related to the "Scope" issue because correct interpretation of "Scope" for DDF rules will also require correct interpretation of entity names (instead of domain names).