Open rufuspollock opened 3 years ago
I think the "multiple files per resource" think is, to the best of my knowledge, not something that is cleanly supported by CKAN. I realize we have some customisations in other projects that allow this, but I'm wondering if is something that should be generically supported by this extension.
I think option B is easy to implement and if we want to support it generically, could be the way to go.
Another option I can suggestion, which is what I consider a cleaner variation of option B, is this:
ckanext-blob-storage
is slightly refactored to offer Python API (e.g. a helper function or similar) that allows downloading anything stored on Giftless as long as we have the organization, dataset, expected sha256 and size of file. It will do the work of authorizing the current user and getting the download link from Giftless, and return it or raise an exception (e.g. if not found or unauthorized). The current download
blueprint will be refactoed to use this. Actually, this is not far from what we have right now. download
URL or not. This will allow ckanext-blob-storage to remain clean and not have code for these kind of special cases.
@shevron agree with you re Option B being the way to go. Also agree that i think it can be changed pretty easily.
I do think it would be worth generifying the default endpoint right now but that's something we can discuss.
How much work do you think it is to do this?
Probably ~1 day but hard to fully estimate without spending some time on analysis. I don't think there is complexity here, just figuring out the right API.
To update, in the latest merge I actually refactored the action that provides download URL / headers to wrap a factored-out Python function (https://github.com/datopian/ckanext-blob-storage/blob/master/ckanext/external_storage/actions.py#L28) that now allows specifying a different sha256 / size / filename for the given resource; This change will allow this extension or other extensions to easily implement the approach I suggested.
Storage bucket:
Uploading files:
Downloading files ...
How CKAN (ckanext-blob-storage) works (?):
url
of a resource: https://ckan.nhs.staging.datopian.com/dataset/65050ec0-5abd-48ce-989d-defc08ed837e/resource/26f3d260-9b90-40c8-90de-c540704f59ac/download/epd_201403Questions:
Proposal
Proposal B:
Modify download handler to be a bit more generic:
Proposal A: download_url API - rejected because we don't want an API ...
resource:
What I want ...
is the ability to store more than one piece of blob data for a resource and get download urls for that