search

nroi / flexo

a central pacman cache
MIT License
172 stars 10 forks source link

Feature Request: Object Storage (S3) #102

Closed benedikt-bartscher closed 1 year ago

benedikt-bartscher commented 1 year ago

It would be great if flexo could store packages in object storage, for example s3. This would further reduce external traffic in clusterized deployments and make it easy to deploy flexo HA.

nroi commented 1 year ago

Can you explain in a bit more detail how you use flexo? I would have expected most if not all users to deploy flexo inside their own four walls in a LAN reachable by a desktop or laptop client. But when you talk about S3, I guess you deploy flexo at AWS or some other cloud provider?

benedikt-bartscher commented 1 year ago

Thanks for your response. I have multiple flexo deployments with helm+k8. Some of them are on-site (local) and some of them in AWS. However every flexo instance is deployed with k8 and 3 pods. Currently these pods use local storage which is not optimal. I would only like to fetch every package 1x from the upstream mirrors to reduce external traffic and increase speed.

nroi commented 1 year ago

I see.

Adding S3 as an additional storage provider would considerably increase the complexity of flexo, since there are some challenges to be addressed. For example, S3 does not support appending data to an existing object. Writing test cases would also become more difficult, because you would have to mock an external provider if you want your tests to run without external dependencies. I'm sure that these are all problems that can be solved, but only with considerable additional complexity.

My assumption is that deploying flexo in an HA environment is more of a niche use-case, hence there are many other issues that provide more benefit for less effort. So this is not on my roadmap in the near future.

Anyone who's interested in this feature: Feel free to comment and argue why you would find this useful, I'm always interested to hear how people use flexo. But since is not on my roadmap at the moment, I'm closing this for now.