search

ckan / datapusher

A standalone web service that pushes data files from a CKAN site resources into its DataStore
GNU Affero General Public License v3.0
77 stars 154 forks source link

Configuration options update #205

Open jbrown-xentity opened 4 years ago

jbrown-xentity commented 4 years ago

This adds a configuration option to not add the CKAN API authorization when using an external storage plugin. Tested in GSA's environment, and all worked as expected. Added documentation for this configuration, as well as some light template for other configuration options.

To fix issue #116

mbocevski commented 4 years ago

@jbrown-xentity which external storage plugin are you using? Cause this should be fixed within that plugin rather then what you are suggesting with this PR. Cause this will only work for public S3 objects. One way to do handle this in the external storage plugin would be to generate presigned URL's for the uploaded resource in S3, like for example https://github.com/keitaroinc/ckanext-s3filestore/blob/master/ckanext/s3filestore/controller.py#L72-L78

amercader commented 4 years ago

@jbrown-xentity is this issue happening with the latest version of DataPusher? This seems to suggest it might had been fixed when refactoring to use requests. If it's still an issue with the external storage plugin you are using, I'd be happy to merge this but with a more specific name for the config option (eg INCLUDE_AUTH_HEADER (default `True.) Also note that when #207 gets merged I added a proper config section in the docs, so the behaviour of the config should be described there.