Feature Request: Support Bulk Import / Mass Ingestion

[theGOTOguy]

I see that HAPI FHIR added a bulk import capability in 5.5.0. The JPA server is apparently still on 5.4.2.

It would be amazing if we had a $import endpoint that could mass-ingest NDJSON in the same format that is emitted by the $export endpoint like Google's FHIR offering does.

[jkiddo]

5.5 isn't released yet

[theGOTOguy]

@jkiddo I don't see a import provider in the 5.5.0 release branch. Do you know if this functionality is planned for 5.5.0?

[jamesagnew]

This functionality has not yet been developed - We have completed much of the prep work but there is no $import provider that will make the cut for 5.5.0 unfortunately.

On Mon, Aug 16, 2021 at 4:12 PM Ben Li-Sauerwine @.***> wrote:

@jkiddo https://github.com/jkiddo I don't see a import provider in the 5.5.0 release branch https://github.com/hapifhir/hapi-fhir/tree/rel_5_5_0/hapi-fhir-jpaserver-base/src/main/java/ca/uhn/fhir/jpa/bulk/imprt. Do you know if this functionality is planned for 5.5.0?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/260#issuecomment-899788738, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2N7HNZYXIWNTMX2RSF6NTT5FWMVANCNFSM5B5LNP4Q . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email .

[theGOTOguy]

@jamesagnew The lack of $import functionality is a major pain point for us right now.

I wonder if the reason it hasn't been implemented is because there is still design work to do, or because you haven't had time in light of other priorities. Would it be acceptable if I submitted a PR implementing a provider in HAPI FHIR and wiring it through to the JPA server?

[RafaelWO]

Any updates on this? According to the changelog it should be available from v6.0 onward. Is there any documentation on how to do a bulk import? Thanks!

[jamesagnew]

For what it's worth, the class

BulkDataImportProviderTest

shows the API surface. We'd gladly accept any PRs to beef up the documentation if you think it's lacking.

Cheers, James

On Wed, Oct 5, 2022 at 2:54 AM RafaelWO @.***> wrote:

Any updates on this? According to the changelog https://hapifhir.io/hapi-fhir/docs/introduction/changelog.html it should be available from v6.0 onward. Is there any documentation on how to do a bulk import? Thanks!

— Reply to this email directly, view it on GitHub https://github.com/hapifhir/hapi-fhir-jpaserver-starter/issues/260#issuecomment-1268024634, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2N7HO3G2ZGNQSO5VNDMUTWBUQ37ANCNFSM5B5LNP4Q . You are receiving this because you were mentioned.Message ID: @.***>

[jkiddo]

@RafaelWO its located here: https://github.com/hapifhir/hapi-fhir/blob/67e67bbf44a76e4c16ebbed307e8313b9c3b2320/hapi-fhir-storage-batch2-jobs/src/test/java/ca/uhn/fhir/batch2/jobs/imprt/BulkDataImportProviderTest.java

[RafaelWO]

Thanks a lot! Am I correct that the bulk import only works with "POSTing" URLs to the resources and not the actual resource content itself?

[janaka44]

Does anyone know if Bulk Import is supported in HAPI open source version ? @RafaelWO @jkiddo ? I'm trying with HAPI FHIR Server - 7.4.0 and get this error message: "Invalid request: The FHIR endpoint on this server does not know how to handle POST operation[$import] with parameters [[]]"

The instructions followed are: https://smilecdr.com/docs/bulk/fhir_bulk_import.html

TIA!

[jkiddo]

@janaka44 have you flipped the https://github.com/hapifhir/hapi-fhir-jpaserver-starter/blob/e2d2ea7726975e76010fc3f98149fad72a48fe84/src/main/resources/application.yaml#L192 ?