google / fhir-data-pipes

A collection of tools for extracting FHIR resources and analytics services on top of that data.
https://google.github.io/fhir-data-pipes/
Apache License 2.0
142 stars 82 forks source link

Pending issues on supporting custom FHIR profiles #960

Closed chandrashekar-s closed 3 months ago

chandrashekar-s commented 5 months ago

Support has been provided to configure extended FHIR profiles as part of this PR. However, currently we support only one custom FHIR profile per resource type due to the limitations that we have for Parquet schema which can have only one schema at a time for the given FHIR Resource type.

Due to the above implementation we have observed the following limitations

  1. Lets say a custom FHIR profile definition internally references to multiple other custom profile definitions(all these referenced custom profiles belong to the same resource type), then due to the limitation that we have of having one profile per resource type, we can configure only one of the referenced profile and need to ignore the other one. This will mandate us to change the definition of the parent profile as well to have reference to only the configured profile type (otherwise the application fails to process the records). This is not great because it is leading to change the definition of the profiles.

For example the us-core-encounter profile depends on multiple us-core-condition profiles as shown below. But we can configure only one of them leading to the change in the definition which is not right. This has to be fixed such that we can configure all the dependent profiles.

Note : If we try to keep the definition of multiple dependent profile in the us-core-encounter file but do not upload the dependent profile definitions, then the application fails to process the Encounter records.

{
  "url": "http://hl7.org/fhir/us/core/StructureDefinition/us-core-encounter",
  "title": "US Core Encounter Profile",
  "type": "Encounter",
  "baseDefinition": "http://hl7.org/fhir/StructureDefinition/Encounter",
  "snapshot": {
    "element": [
      {
        "id": "Encounter.reasonReference",
        "path": "Encounter.reasonReference",
        "type": [
          {
            "code": "Reference",
            "targetProfile": [
              "http://hl7.org/fhir/us/core/StructureDefinition/us-core-condition-problems-health-concerns",
              "http://hl7.org/fhir/us/core/StructureDefinition/us-core-condition-encounter-diagnosis",
              "http://hl7.org/fhir/us/core/StructureDefinition/us-core-procedure",
              "http://hl7.org/fhir/StructureDefinition/Observation",
              "http://hl7.org/fhir/StructureDefinition/ImmunizationRecommendation"
            ]
          }
        ]
      }
    ]
  }
}
  1. In case of FHIR Profiling, the extended profile can apply constraints on the base fields. Lets say in a base profile a parameter is given flexibility to taken multiple data types, then in the extended profile we could limit this to a certain type and apply additional constraints. In such cases, if we want to support the extended profile then the flexibility to support other types gets lost.

For example, the base Observation profile has the Observation.value[x] field which can take types like [Quantity, CodeableConcept, String, Boolean etc] and in the extended Us Core Observation Occupation Profile the field Observation.value[x] can take only CodeableConcept. Because of this, if we want to process a Record belonging to a base profile or other profiles (having other data types) we loose the flexibility as during conversion the extended profile with support for only one type gets applied and the other data type fields are lost during conversion.

chandrashekar-s commented 4 months ago

As part of this PR #924, the first issue is partially addressed now i.e. we accept all Structure definitions for the same resource and lexicographically choose the smaller profile url (which is always deterministic for the given set of definitions), which can be used for Avro conversions. A more stable solution will be provided by #980.

bashir2 commented 3 months ago

@chandrashekar-s now that #986 is merged and #980 is fixed, can we close this issue too?

chandrashekar-s commented 3 months ago

The issues mentioned in this ticket have been resolved as part of the PR #986, i.e. now we can configure multiple profiles against the same resource type and consider all the extensions defined by all the profiles.