jamesagnew
commented
4 years ago The IBinaryStorageSvc interface is intended for exactly this kind of thing- It offloads the binary contents out of the DB into whatever implementation of the interface is provided.
Currently only a filesystem implementation exists, but it should be possible to create one backed by S3. PRs welcomed!
Describe the issue Instead of storing bulk-exported ndjson files on a HAPI FHIR server, there should be an option to write these directly to external object storage (such as S3, Ceph, Minio, etc.). This should reduce the load on the server and will store the files in a way that is better suited for subsequent processing.
Additional context As the de-facto standard for object storage, implementing a S3-API-based bulk export service provider would support the largest number of external storage providers (besides AWS).