search

datopian / ckanext-asset-storage

CKAN extension implementing IUploader interface for cloud storage of assets e.g. images. Intended for use with ckanext-blob-storage that provides direct to cloud storage of resource data.
MIT License
2 stars 2 forks source link

Investigate using libcloud to add additional storage providers #5

Closed shevron closed 4 years ago

shevron commented 4 years ago

It may be simpler to just implement a single libcloud backend to support many different cloud vendors. Let's look into it, so we can get all cloud providers for the price of one!

shevron commented 4 years ago

From what I can tell, libcloud doesn't abstract things like making objects public / private (ACL) or getting signed URLs for private objects very well. If this capability is provided (and it doesn't seem to be provided or documented for all major vendors, e.g. don't see how to do it in Google Cloud), the API differs from vendor to vendor.

Since implementing this for each vendor is probably going to be not that hard (per vendor), and if using libcloud will require us to do some vendor-specific hacks anyway, I would prefer to just implement an Azure and S3 driver and be done with it.

We can probably do a libcloud driver that doesn't support public assets at all, but that would be a shame so I would only do it as a "bonus" and not instead of vendor-specific backends for the major vendors.

shevron commented 4 years ago

/cc @rufuspollock closing this for now in favor of implementing an Azure specific adapter.