search

TkTech / ckanext-cloudstorage

Implements support for resource storage against multiple popular providers via apache-libcloud (S3, Azure Storage, etc...)
MIT License
35 stars 55 forks source link

Configuration option to expose S3 url #26

Closed jqnatividad closed 4 years ago

jqnatividad commented 6 years ago

This will enable tools like Carto to use the resource URL to load data.

Right now, if I give Carto the resource URL, it gets a HTTP 301 (redirect) which stops it from loading data. This is being done to do a "visualize with Carto..." link.

Optionally exposing the S3 URL should allow users to directly use AWS services that expect an S3 URL like Athena, Lambda, etc.

TkTech commented 6 years ago

@jqnatividad can these tools not be configured to follow redirects? Directly exposing the S3 URL as the download url means you would lose all CKAN-imposed access restrictions and the assets on your bucket must be public.

It might be best to instead add a separate field in your schema for direct downloads (possibly a hidden field), populating it in your plugin after the upload completes. Trigger an ACL update for that object on S3 to allow public access. This way you can still have public & private resources with access restrictions while exposing things selectively.