Bulk Data Group/$export using Group.characteristic

[lmsurpre]

follow-on from #108

1. review the cohorts we want to support
2. express the cohorts using the FHIR Group resource (using inclusion and exclusion criteria)
3. write generic utility for resolving the inclusion/exclusion criteria into a list of patient resources
4. hook it into the bulk export operation

Suggestions for making this tractable:

1. start with export on just a single criteria (e.g. look at the Group examples)
2. alternative: start with just the the 4 characteristics defined at https://www.hl7.org/fhir/population.html
3. manually construct the search string that corresponds to a single criteria

Then: identify the delta between that and a more complicated cohort definition like one for Diabetes.

[prb112]

Related to FHIR Resource: Group.characteristic Example JSON with Characteristic

[prb112]

Source: https://ibm.ent.box.com/file/922194231982

The HL7 FHIR Bulk Data Export Implementation Guide has support for the Group/$export extended operation. The Group/$export exports all user data pertaining to a group (in the IBM FHIR Server it’s limited to the Patient Compartment data).

There are two main types of groups Descriptive or Actual; specified in the Group.actual field. An actual group specifies a set of members Group.member where each member includes a Reference (Patient/1-2-3-4). A descriptive group uses Include/Exclude traits that are described as a combination of FHIR Data Types:

valueCodeableConcept CodeableConcept valueBoolean boolean valueQuantity Quantity valueRange Range valueReference Reference

Today, the IBM FHIR Server supports Actual (Group.member). The team has implemented the processing of the Dynamic Groups for specific use cases, such as Age Range-Blood Pressure. The package com.ibm.fhir.operation.bulkdata.tool.helpers.dynamic contains sample groups and Patient Bundles in Java code and actual JSONs in fhir-examples/src/main/resources/json/ibm/bulk-data/group.

An example characteristic is:

"characteristic": [
        {
            "code": {
                "coding": [
                    {
                        "system": "http://loinc.org",
                        "code": "29553-5"
                    }
                ],
                "text": "Age calculated"
            },
            "valueQuantity": {
                "value": 13,
                "comparator": ">=",
                "unit": "years",
                "system": "http://unitsofmeasure.org",
                "code": "a"
            },
            "exclude": false
        }]

The team built a GroupSearchCompiler (implementation: GroupSearchCompilerImpl) in fhir-search which takes the Group and uses characteristic specific processing to compile down to search parameters:

private static final List<CharacteristicProcessor> CODEABLE_CONCEPT_PROCESSORS = Arrays.asList(new PregnancyStatusCharacteristicProcessor(), new AdministrativeGenderCharacteristicProcessor());
private static final List<CharacteristicProcessor> BOOLEAN_PROCESSORS = Arrays.asList(new NoOpCharacteristicProcessor());
private static final List<CharacteristicProcessor> RANGE_PROCESSORS = Arrays.asList(new NoOpCharacteristicProcessor());
private static final List<CharacteristicProcessor> REFERENCE_PROCESSORS = Arrays.asList(new NoOpCharacteristicProcessor());
private static final List<CharacteristicProcessor> QUANTITY_PROCESSORS = Arrays.asList(new PatientAgeCharacteristicProcessor());

The generated Search is targeted on Patient in order to generate a list of patients. For example an Age search would generate a birthdate=le2000&birthdate=ge1990 for a target age range.

The next steps to complete the Dynamic Group is to hook into the com.ibm.fhir.bulkdata.export.group.resource.GroupHandler.process(groupId), and check to see if the group retrieved is actual or characteristic. If Group.actual = false and there are characteristics, then process GroupSearchCompiler to get a Search Parameter, and then Page through the Search each time it is called.

public void process(String groupId) throws Exception {
        if (patientMembers == null) {
            Group group = findGroupByID(groupId);
            uniquenessGuard = new HashSet<>();
            patientMembers = new ArrayList<>();
            // List for the group and sub groups in the expansion paths, this is used to avoid dead loop caused by circle reference of the groups.
            Set<String> groupsInPath = new HashSet<>();
            expandGroupToPatients(group, groupsInPath);
        }
    }