pbinkley
commented
9 years ago Strategies for OpenStack Swift integration:
- Swift as object store for Fedora: files are ingested into Swift on upload and fetched from Swift when needed. The only local cache is used between upload and ingest.
- Pro: less local storage needed
- Con: needs significant work at Fedora level (though others may be doing / have done it); introduces critical dependency on Swift (nothing can be served when Swift is down); performance issues when serving lots of files from Swift unknown, and likely to be exacerbated when Peel moves into Hydra
- possibilities: JCloud module
- Object Store in local storage, and ingest into Swift is triggered and performed like other post-upload tasks. Files are served from local storage.
- Pro: no dependence on Swift for serving content; less development work.
- Con: requires large and growing local cache, which will become quite large with Peel if we go to a tiles-everywhere approach to serving images
- we could mitigate local storage needs by implementing a true local cache, where objects are fetched from Swift when needed, and kept in local storage until they expire.
- Notes
- the Swift as object store strategy is essentially off the table until the Unix team is ready to support Swift to the level necessary to provide the necessary uptime guarantee
- Implementing one strategy will not make it difficult to migrate to the other
- Current use of Apache Camel in Fedora 4 would make the triggered ingest easy to implement
- Geoff's priorities
- Tiles everywhere
- Ability for users to download high-res images directly (we will stop charging for this when it is automated)
- One big question: performance when retrieving multimedia service copies from Swift to feed a streaming server
This work has moved to the pushmi-pullyu repo