search

mercure-imaging / mercure

mercure DICOM Orchestrator
https://mercure-imaging.org
MIT License
65 stars 31 forks source link

S3 Support #76

Closed cadrogui closed 3 months ago

cadrogui commented 3 months ago

Hi everyone, maintain DICOM files in local FS or SFS is very expensive, so in you roadmap there is any plans to integrante S3, via wrapper like Apache jclouds, or AWS SDK?, if not there's any way for acomplish that using the modular architecture of mercure?

Regards.

guruevi commented 3 months ago

Hi, Mercure is a DICOM router, it doesn't/shouldn't store much data locally (except for the cache). You could put the cache on S3 using something like S3FS though or if you're running the system in Docker/Kubernetes, there are plugins to do the same.

However, having individual DICOM files in S3 can be very expensive (because of the per-object/per-transaction cost), I would not recommend it.

If you're looking for data organization, we are using XNAT for that, we have worked on some plugins to send data to S3 in there for long-term data storage/archival purposes. But if you're looking at a PACS, various open source PACS exist, some also have support or plugins for S3.

tblock79 commented 3 months ago

As guruevi explained, mercure is not intended for holding large amounts of files. It is a router / batch processor and not a PACS. Once the processing and/or dispatching of a study has been completed, the DICOMs will be discarded. Local diskspace is only needed for buffering images. If you are running out of local disk space, you can reduce the retention period, so that images will be discarded earlier. Using S3 for buffering the DICOMs is not recommended as it would make the router very slow.