Format for commiting Quantity data type in FLAT composition

dileep-ehr commented 3 years ago

Configuration information

0.16.5

<openehr_sdk_version>39cb1fd</openehr_sdk_version>
<archie_version>5a4a9442e5e2516aed5f25636ac236682aace7b9</archie_version>
<jvm_version>Oracle Corporation 11.0.11+9</jvm_version>
<os_version>Linux amd64 5.4.0-74-generic</os_version>
<postgres_version>PostgreSQL 11.5 on x86_64-pc-linux-musl, compiled by gcc (Alpine 8.3.0) 8.3.0, 64-bit</postgres_version>

Steps to reproduce

Quantity data type value in FLAT format

"vital_signs/vital_signs:0/blood_pressure:0/any_event:0/systolic|magnitude": 120.0, "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/systolic|unit": "mm[Hg]"

Actual result

The server seems to hang and does not give any response even after waiting for a long time. The commit goes through if this is removed

Is this the correct format to add suffixes to a quantity node that has both magnitude & unit suffixes? Ethercis supported this method

Expected result (Acceptance Criteria)

200 OK.

Definition of Done

[ ] The defect is checked by an unit or an integration test (Robot)
[ ] Merge Request approved
[ ] Unit tests passed
[ ] Build without errors
[ ] Release notes prepared
[ ] No additional runtime warnings

dileep-ehr commented 3 years ago

OPT attached for reference EHRN Vital signs.v1.zip

subigre commented 3 years ago

Dear @dileep-ehr,

I tried to have a look and it seems that is the same cause than the issue reported in this comment.

I am able to create a composition successfully using the JSON format:

{
    "_type": "COMPOSITION",
    "name": {
        "_type": "DV_TEXT",
        "value": "EHRN Vital signs.v2"
    },
    "archetype_details": {
        "_type": "ARCHETYPED",
        "archetype_id": {
            "_type": "ARCHETYPE_ID",
            "value": "openEHR-EHR-COMPOSITION.encounter.v1"
        },
        "template_id": {
            "_type": "TEMPLATE_ID",
            "value": "EHRN Vital signs.v2"
        },
        "rm_version": "1.0.4"
    },
    "archetype_node_id": "openEHR-EHR-COMPOSITION.encounter.v1",
    "language": {
        "_type": "CODE_PHRASE",
        "terminology_id": {
            "_type": "TERMINOLOGY_ID",
            "value": "ISO_639-1"
        },
        "code_string": "fr"
    },
    "territory": {
        "_type": "CODE_PHRASE",
        "terminology_id": {
            "_type": "TERMINOLOGY_ID",
            "value": "ISO_3166-1"
        },
        "code_string": "FR"
    },
    "category": {
        "_type": "DV_CODED_TEXT",
        "value": "event",
        "defining_code": {
            "_type": "CODE_PHRASE",
            "terminology_id": {
                "_type": "TERMINOLOGY_ID",
                "value": "openehr"
            },
            "code_string": "433"
        }
    },
    "composer": {
        "_type": "PARTY_IDENTIFIED",
        "name": "Renaud"
    },
    "context": {
        "_type": "EVENT_CONTEXT",
        "start_time": {
            "_type": "DV_DATE_TIME",
            "value": "2021-03-21T21:19:49.293785+01:00"
        },
        "setting": {
            "_type": "DV_CODED_TEXT",
            "value": "other care",
            "defining_code": {
                "_type": "CODE_PHRASE",
                "terminology_id": {
                    "_type": "TERMINOLOGY_ID",
                    "value": "openehr"
                },
                "code_string": "238"
            }
        }
    },
    "content": [
        {
            "_type": "SECTION",
            "name": {
                "_type": "DV_TEXT",
                "value": "Vital signs"
            },
            "archetype_details": {
                "_type": "ARCHETYPED",
                "archetype_id": {
                    "_type": "ARCHETYPE_ID",
                    "value": "openEHR-EHR-SECTION.vital_signs.v1"
                },
                "rm_version": "1.0.4"
            },
            "archetype_node_id": "openEHR-EHR-SECTION.vital_signs.v1",
            "items": [
                {
                    "_type": "OBSERVATION",
                    "name": {
                        "value": "Blood pressure",
                        "_type": "DV_TEXT"
                    },
                    "subject": {
                        "_type": "PARTY_SELF"
                    },
                    "encoding": {
                        "_type": "CODE_PHRASE",
                        "code_string": "UTF-8",
                        "terminology_id": {
                            "_type": "TERMINOLOGY_ID",
                            "value": "IANA_character-sets"
                        }
                    },
                    "language": {
                        "_type": "CODE_PHRASE",
                        "code_string": "de",
                        "terminology_id": {
                            "_type": "TERMINOLOGY_ID",
                            "value": "ISO_639-1"
                        }
                    },
                    "data": {
                        "_type": "HISTORY",
                        "name": {
                            "_type": "DV_TEXT",
                            "value": "History"
                        },
                        "origin": {
                            "_type": "DV_DATE_TIME",
                            "value": "2021-03-21T21:19:49.293785+01:00"
                        },
                        "events": [
                            {
                                "_type": "POINT_EVENT",
                                "name": {
                                    "_type": "DV_TEXT",
                                    "value": "Any event"
                                },
                                "archetype_node_id": "at0006",
                                "time": {
                                    "_type": "DV_DATE_TIME",
                                    "value": "2021-03-21T21:19:49.293785+01:00"
                                },
                                "data": {
                                    "_type": "ITEM_TREE",
                                    "name": {
                                        "_type": "DV_TEXT",
                                        "value": "blood pressure"
                                    },
                                    "archetype_node_id": "at0003",
                                    "items": [
                                        {
                                            "_type": "ELEMENT",
                                            "name": {
                                                "_type": "DV_TEXT",
                                                "value": "Systolic"
                                            },
                                            "archetype_node_id": "at0004",
                                            "value": {
                                                "_type": "DV_QUANTITY",
                                                "magnitude": "120.0",
                                                "units": "mm[Hg]"
                                            }
                                        }
                                    ]
                                }
                            }
                        ],
                        "archetype_node_id": "at0001"
                    },
                    "archetype_node_id": "openEHR-EHR-OBSERVATION.blood_pressure.v2"
                }
            ]
        }
    ]
}

However, I have the infinite loop if I used the exact same composition in JSON Flat format:

{
    "ehrn_vital_signs.v2/language|terminology": "ISO_639-1",
    "ehrn_vital_signs.v2/language|code": "fr",
    "ehrn_vital_signs.v2/composer|name": "Renaud",
    "ehrn_vital_signs.v2/territory|terminology": "ISO_3166-1",
    "ehrn_vital_signs.v2/territory|code": "FR",
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/any_event:0/systolic|magnitude": 120.0,
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/any_event:0/systolic|unit": "mm[Hg]",
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/any_event:0/time": "2021-03-21T21:19:49.293785+01:00",
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/encoding|terminology": "IANA_character-sets",
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/encoding|code": "UTF-8",
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/language|terminology": "ISO_639-1",
    "ehrn_vital_signs.v2/vital_signs:0/blood_pressure:0/language|code": "de",
    "ehrn_vital_signs.v2/context/setting|terminology": "openehr",
    "ehrn_vital_signs.v2/context/setting|code": "238",
    "ehrn_vital_signs.v2/context/setting|value": "other care",
    "ehrn_vital_signs.v2/context/start_time": "2021-03-21T20:19:49.293+01:00",
    "ehrn_vital_signs.v2/category|value": "event",
    "ehrn_vital_signs.v2/category|terminology": "openehr",
    "ehrn_vital_signs.v2/category|code": "433"
}

It seems that the issue is the consequence of the two following causes:

The FlatJsonUnmarshaller creates a Composition with an additional INTERVAL_EVENT (which is not populated).

The version of the archie library used in the openEHR_SDK goes into the infinite loop because parent is never updated if attributeParent == null (this issue is now fixed in the last versions)

  public static String getParentObservationTerm(CAttribute attribute) {
        String result = "";
        CObject parent = attribute.getParent();
        while (result.equals("") && parent != null) {
            CAttribute attributeParent = parent.getParent();
            if (attributeParent != null) {
                parent = attributeParent.getParent();
                if (parent.getRmTypeName().equals("OBSERVATION")) {
                    ArchetypeTerm parentTerm = parent.getTerm();
                    if (parentTerm != null) {
                        result = parentTerm.getText();
                    }
                }
            }
        }
        return result;
    }

dileep-ehr commented 3 years ago

Is the updated code available in develop branch to test now?

dileep-ehr commented 3 years ago

Tested the develop branch and the problem is still there. Also the Archie library version is the same

sidharthramesh commented 3 years ago

The following composition works with the EHRN Vital signs.v1 template:

{
        "vital_signs/category|code": "433",
        "vital_signs/category|value": "event",
        "vital_signs/category|terminology": "openehr",
        "vital_signs/context/ehrn_metadata:0/confidentiality_level|magnitude": 31,
        "vital_signs/context/start_time": "2021-07-05T11:17:40.788Z",
        "vital_signs/context/setting|code": "238",
        "vital_signs/context/setting|value": "other care",
        "vital_signs/context/setting|terminology": "openehr",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/systolic|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/systolic|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/diastolic|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/diastolic|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/mean_arterial_pressure|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/mean_arterial_pressure|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/pulse_pressure|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/pulse_pressure|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/position|code": "at1000",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/position|value": "Standing",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/position|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/sleep_status|code": "at1044",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/sleep_status|value": "Awake",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/sleep_status|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/tilt|unit": "deg",
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/tilt|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/any_event:0/time": "2021-07-05T11:17:40.788Z",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/math_function|code": "146",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/math_function|value": "mean",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/math_function|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/systolic|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/systolic|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/diastolic|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/diastolic|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/mean_arterial_pressure|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/mean_arterial_pressure|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/pulse_pressure|unit": "mm[Hg]",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/pulse_pressure|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/position|code": "at1000",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/position|value": "Standing",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/position|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/sleep_status|code": "at1044",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/sleep_status|value": "Awake",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/sleep_status|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/tilt|unit": "deg",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/tilt|magnitude": 31,
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/time": "2021-07-05T11:17:40.788Z",
        "vital_signs/vital_signs:0/blood_pressure:0/cuff_size|code": "at0015",
        "vital_signs/vital_signs:0/blood_pressure:0/cuff_size|value": "Adult Thigh",
        "vital_signs/vital_signs:0/blood_pressure:0/cuff_size|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/location_of_measurement|code": "at0025",
        "vital_signs/vital_signs:0/blood_pressure:0/location_of_measurement|value": "Right arm",
        "vital_signs/vital_signs:0/blood_pressure:0/location_of_measurement|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/method|code": "at1036",
        "vital_signs/vital_signs:0/blood_pressure:0/method|value": "Auscultation",
        "vital_signs/vital_signs:0/blood_pressure:0/method|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/diastolic_endpoint|code": "at1011",
        "vital_signs/vital_signs:0/blood_pressure:0/diastolic_endpoint|value": "Phase IV",
        "vital_signs/vital_signs:0/blood_pressure:0/diastolic_endpoint|terminology": "local",
        "vital_signs/vital_signs:0/blood_pressure:0/language|code": "en",
        "vital_signs/vital_signs:0/blood_pressure:0/language|terminology": "ISO_639-1",
        "vital_signs/vital_signs:0/blood_pressure:0/encoding|code": "UTF-8",
        "vital_signs/vital_signs:0/blood_pressure:0/encoding|terminology": "IANA_character-sets",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/rate|unit": "/min",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/rate|magnitude": 31,
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/regular|code": "at0006",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/regular|value": "Regular",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/regular|terminology": "local",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/irregular_type|code": "at0007",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/irregular_type|value": "Regularly Irregular",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/irregular_type|terminology": "local",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/any_event:0/time": "2021-07-05T11:17:40.788Z",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/method|code": "at1032",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/method|value": "Palpation",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/method|terminology": "local",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/body_site|code": "at1038",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/body_site|value": "Radial Artery - Left",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/body_site|terminology": "local",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/language|code": "en",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/language|terminology": "ISO_639-1",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/encoding|code": "UTF-8",
        "vital_signs/vital_signs:0/pulse_heart_beat:0/encoding|terminology": "IANA_character-sets",
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/temperature|unit": "Cel",
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/temperature|magnitude": 31,
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/body_exposure|code": "at0031",
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/body_exposure|value": "Naked",
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/body_exposure|terminology": "local",
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/current_day_of_menstrual_cycle|magnitude": 31,
        "vital_signs/vital_signs:0/body_temperature:0/any_event:0/time": "2021-07-05T11:17:40.789Z",
        "vital_signs/vital_signs:0/body_temperature:0/location_of_measurement|code": "at0025",
        "vital_signs/vital_signs:0/body_temperature:0/location_of_measurement|value": "Rectum",
        "vital_signs/vital_signs:0/body_temperature:0/location_of_measurement|terminology": "local",
        "vital_signs/vital_signs:0/body_temperature:0/language|code": "en",
        "vital_signs/vital_signs:0/body_temperature:0/language|terminology": "ISO_639-1",
        "vital_signs/vital_signs:0/body_temperature:0/encoding|code": "UTF-8",
        "vital_signs/vital_signs:0/body_temperature:0/encoding|terminology": "IANA_character-sets",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/rate|unit": "/min",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/rate|magnitude": 31,
        "vital_signs/vital_signs:0/respirations:0/any_event:0/rhythm|code": "at0006",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/rhythm|value": "Regular",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/rhythm|terminology": "local",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/depth|code": "at0017",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/depth|value": "Normal",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/depth|terminology": "local",
        "vital_signs/vital_signs:0/respirations:0/any_event:0/time": "2021-07-05T11:17:40.789Z",
        "vital_signs/vital_signs:0/respirations:0/language|code": "en",
        "vital_signs/vital_signs:0/respirations:0/language|terminology": "ISO_639-1",
        "vital_signs/vital_signs:0/respirations:0/encoding|code": "UTF-8",
        "vital_signs/vital_signs:0/respirations:0/encoding|terminology": "IANA_character-sets",
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spo|numerator": 31,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spo|denominator": 100,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spo|type": 2,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spoc|unit": "ml/dl",
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spoc|magnitude": 31,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spco|numerator": 31,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spco|denominator": 100,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spco|type": 2,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spmet|numerator": 31,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spmet|denominator": 100,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/spmet|type": 2,
        "vital_signs/vital_signs:0/pulse_oximetry:0/any_event:0/time": "2021-07-05T11:17:40.789Z",
        "vital_signs/vital_signs:0/pulse_oximetry:0/pre_post-ductal|code": "at0062",
        "vital_signs/vital_signs:0/pulse_oximetry:0/pre_post-ductal|value": "Pre-ductal",
        "vital_signs/vital_signs:0/pulse_oximetry:0/pre_post-ductal|terminology": "local",
        "vital_signs/vital_signs:0/pulse_oximetry:0/language|code": "en",
        "vital_signs/vital_signs:0/pulse_oximetry:0/language|terminology": "ISO_639-1",
        "vital_signs/vital_signs:0/pulse_oximetry:0/encoding|code": "UTF-8",
        "vital_signs/vital_signs:0/pulse_oximetry:0/encoding|terminology": "IANA_character-sets",
        "vital_signs/composer|name": "Medblocks UI",
        "vital_signs/language|code": "en",
        "vital_signs/language|terminology": "ISO_639-1",
        "vital_signs/territory|code": "IN",
        "vital_signs/territory|terminology": "ISO_3166-1",
        "vital_signs/vital_signs:0/blood_pressure:0/a24_hour_average/width": "P3DT10H56M"
}

I automatically generated this as part of a stress test that creates every possible FLAT composition for any given template. Been testing against EHRBase and the infinite loop problem does come up a lot. I don't want to spam the issues here with thousands of compositions though.

dileep-ehr commented 3 years ago

I think I have narrowed the problem to the interval clusters in BP and Pulse archetypes. Though these have 0::1 occurrence, server is expecting values in them and gets into a loop if they are empty. Constraining this cluster out from template seems to fix the problem.

This probably can be generalized to any archetype with an optional interval cluster

ehrbase / ehrbase