search

google / fhir-data-pipes

A collection of tools for extracting FHIR resources and analytics services on top of that data.
https://google.github.io/fhir-data-pipes/
Apache License 2.0
151 stars 84 forks source link

The pipeline fails with `Out of Heap Space` error for high number of FHIR resources #900

Closed chandrashekar-s closed 7 months ago

chandrashekar-s commented 10 months ago

In a limited machine resource environment, the pipeline fails with Out of Heap Space error for the Incremental Run when processing a huge number of records.

For example, the Incremental Run failed in the below scenario

Cores: 24
Java Heap Space (-Xmx): 4g
Initially loaded number of Patient resources: 9K
Incrementally added number of Patient resources : 500

The Incremental run failed with Out of Heap Space error in the above scenario. Error to be attached