Add object store backend to further enable scalable cloud deployments

MisterTimn commented 2 years ago

Pitch

Object storage is a go to storage backend in many cloud based use cases. Cloud providers such as AWS and GKE offer scalable and affordable object storage options. CSS should have an optional Object Storage backend so cloud deployments are made easy and effective. Also with regard to scalability, Object storage has an upper hand compared to other storage forms like file or block.

Desired solution

Build out the object storage backend (using self deployed MinIO and later moving to S3 for instance)
Maximize the usage of Object Storage, as it is very unrestrictive and optimizable (no hierarchy, full control of metadata).

Acceptance criteria

CSS can use MinIO and/or other S3 compliant Object Stores
CSS can be deployed on AWS and use S3 Storage
CSS can be deployed on GKE and use Google Object Storage
Installation instructions for these use-cases are provided (through e.g. Helm chart, shell scripts, etc.)

pheyvaer commented 2 years ago

@RubenVerborgh What changes do you expect here?

RubenVerborgh commented 2 years ago

A challenge is a technical problem applied to a concrete use case with as output a demo. I'm missing the use case and demo parts.

pheyvaer commented 2 years ago

@MisterTimn Did you have the chance to look into making the necessary changes?

MisterTimn commented 2 years ago

Not yet, will have to take some time to further investigate and experiment with it before I can properly define a concrete use case and demo

pheyvaer commented 2 years ago

@MisterTimn I want to check in to see if you already had the chance to further investigate.

MisterTimn commented 1 year ago

No and probably won't be able to any time soon... should I just close the challenge for now?

SolidLabResearch / Challenges