google / fhir-data-pipes

A collection of tools for extracting FHIR resources and analytics services on top of that data.
https://google.github.io/fhir-data-pipes/
Apache License 2.0
141 stars 80 forks source link

Added support to read FHIR resources from NDJson file #1009

Closed chandrashekar-s closed 2 months ago

chandrashekar-s commented 3 months ago

Description of what I changed

Fixes #362 Added support for reading FHIR resources from NDJson file.

E2E test

Added unit tests

TESTED:

Tested end-to-end by running the pipeline for ndjson files using the following command and verifed if the parquet files are created correctly.

    com.google.fhir.analytics.FhirEtl \
    --outputParquetPath=/tmp/TEST/ \
    --sourceNDJsonFilePattern=/tmp/ndjson/* \
    --resourceList=Patient,Encounter,Observation

Checklist: I completed these to help reviewers :)

chandrashekar-s commented 2 months ago

Thanks @bashir2 for the review, I have addressed the review comments in the latest commit. Please have a look.