Support for custom FHIR profile

[chandrashekar-s]

Description of what I changed

• The user can load the definitions of custom profiles in a directory and the application will load these structure definitions during launch.
• The application overrides the default profiles for the resources with the ones loaded by the user. Currently, only one custom profile is supported for a resource type. In case if a custom profile exists for a resource type, then it will take precedence over the default profile. In case of multiple custom profiles for the same resource type, the custom profile url that is lexicographically smaller will be chosen.
• While converting the FHIR resources to Avro records, the overloaded profiles will be applied on the FHIR resources irrespective of which FHIR profile they belong to.

Issues Fixed:

560

400

753

558

Pending Changes:

In the long term, we need to provide support for multiple custom profiles. This is being tracked in #980 The issues mentioned in #960 and #961 are partially fixed in the current PR.

E2E test

The application was loaded with custom us-core-profile definitions and the pipeline was run. The tables that were created in the spark thrift server contained the extended definitions defined in the us-core-profile.

TESTED:

Please replace this with a description of how you tested your PR beyond the automated e2e/unit tests.

Checklist: I completed these to help reviewers :)

• [x] I have read and will follow the review process.

• [x] I am familiar with Google Style Guides for the language I have coded in.

  No? Please take some time and review Java and Python style guides.

• [x] My IDE is configured to follow the Google code styles.

  No? Unsure? -> configure your IDE.

• [x] I have added tests to cover my changes. (If you refactored existing code that was well tested you do not have to add tests)

• [x] I ran mvn clean package right before creating this pull request and added all formatting changes to my commit.

• [x] All new and existing tests passed.

• [x] My pull request is based on the latest changes of the master branch.

  No? Unsure? -> execute command git pull --rebase upstream master

[chandrashekar-s]

@bashir2 The changes have been tested for the Patient resource with us-core-profile. For other resources it was failing because of existing issues in bunsen, will file a bug for the same. Also, for supporting multiple profiles will file another bug.

[chandrashekar-s]

@bashir2 In the latest commit, I have added support for configuring fhir profiles via classpath names. By default, the classpath name containing the us-core fhir profiles have been configured, but user can override them via a different directory as well.

Also, all the us-core fhir profiles definitions have been moved to a common package and is being referenced in all the test cases now.

There are few issues which are open and are being tracked via #960 and #961.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 53.41246% with 157 lines in your changes are missing coverage. Please review.

Project coverage is 49.61%. Comparing base (ce08411) to head (f39fbc7).

Files	Patch %	Lines
...java/com/cerner/bunsen/ProfileMappingProvider.java	66.96%	31 Missing and 6 partials :warning:
...ava/com/google/fhir/analytics/PipelineManager.java	18.75%	26 Missing :warning:
.../com/google/fhir/analytics/AvroConversionUtil.java	55.10%	16 Missing and 6 partials :warning:
...c/main/java/com/google/fhir/analytics/FhirEtl.java	0.00%	19 Missing :warning:
...a/com/cerner/bunsen/ProfileMapperFhirContexts.java	67.34%	14 Missing and 2 partials :warning:
.../java/com/google/fhir/analytics/ParquetMerger.java	0.00%	15 Missing :warning:
.../main/java/com/google/fhir/analytics/EtlUtils.java	0.00%	9 Missing :warning:
...erner/bunsen/definitions/StructureDefinitions.java	72.22%	2 Missing and 3 partials :warning:
...erner/bunsen/exception/ProfileMapperException.java	0.00%	2 Missing :warning:
...m/google/fhir/analytics/ProcessGenericRecords.java	0.00%	2 Missing :warning:
... and 3 more

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #924 +/- ## ============================================ - Coverage 50.10% 49.61% -0.50% - Complexity 605 631 +26 ============================================ Files 86 87 +1 Lines 5073 5196 +123 Branches 632 663 +31 ============================================ + Hits 2542 2578 +36 - Misses 2285 2357 +72 - Partials 246 261 +15 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[bashir2]

Thanks @chandrashekar-s for the changes; is this ready for another round of review? I mean, are the changes we discussed last week applied?

[chandrashekar-s]

Thanks @chandrashekar-s for the changes; is this ready for another round of review? I mean, are the changes we discussed last week applied?

There are some minor documentation changes pending, will update once done (should be done today).

[chandrashekar-s]

@bashir2 In the latest commit, I have partially addressed the issues listed in #960 and #961. I have also created the issue #980 to create a generic solution for multiple extensions. Updated the description of the PR to reflect the current implementation status.

You can review the last 2 commits now.

[chandrashekar-s]

@bashir2 In the latest commit, I have partially addressed the issues listed in #960 and #961. I have also created the issue #980 to create a generic solution for multiple extensions. Updated the description of the PR to reflect the current implementation status.

You can review the last 2 commits now.

The description also contains a Pending Items section which will be addressed later on, not in this PR.

[bashir2]

Side note1: The PRs Fixed section of the description should probably be Issues Fixed but even with that I am not sure if that causes merging this PR to close those issues.

Side note2: It is great to see how many issues are being resolved by this PR; great job.