[STORY] Investigate S3 storage solutions

Australian-Imaging-Service / pipelines

Scripts to generate analysis pipelines that can be run in XNAT's container service

Other

1 stars 3 forks source link

Open tclose opened 3 weeks ago

tclose commented 3 weeks ago

Epic: #271 Feature: Feature Release: Required knowledge: shallow or mid-level or deep

Need to find a cheaper, reasonably performant storage solution for cloud object storage (e.g. S3 file-gateway, Objective FS)

[ ] 1. Settle on a solution that will be able to store all USyd's data requirements for next couple of years (e.g. TBP)
[ ] 2. Layout on transition plan as large-scale university storage moves to local cloud

tclose commented 1 week ago

Cheapest option is S3 file gateway, no auto-scaling/high availability
FSX (Lustre) - good but expensive
FS "on-tap" - similar to FSX but cheaper, still more expensive than S3 file-gateway, but might be the best option