cdisc-org / cdisc-rules-engine

Open source offering of the cdisc rules engine
MIT License
45 stars 12 forks source link

Invalid JSON returned when using `record_count` operation with `group` and `filter` #705

Open ASL-rmarshall opened 1 month ago

ASL-rmarshall commented 1 month ago

If both group and filter are used with the record_count operator, it is possible for a count result to be missing for one or more of the groups (because there are no records that meet the filter criteria for those groups). When the result of the record_count is merged back onto the main dataset, the record_count "$id" column will contain NaN values when there is no record count result associated with the group. If the record_count "$id" column is included in the list of Outcome / Output Variables, this causes a 500 - Core Engine returned invalid JSON error in the rules editor.

Example rule: DDF99912 in the test environment Test data: You can use the DDF00012 negative test data in USDM subfolder of the CORE unitTesting SharePoint

To prevent this error, before returning the result of the operation, you could consider merging the record count results back onto a full list of groups (without filtering) and replace any NaNs with zero (i.e., there were no records for that group). I'm not sure what happens if you use filtering without groups - if there are no records that match the filter, do you get zero or NaN? If NaN is returned it would be better to again replace with zero.

Also, please update the documentation for the record_count operation to: