Closed rufuspollock closed 3 years ago
Script has been merged in #40. Note that we do download and upload and not use any Azure APIs because we don't want to have anything Azure specific. We use CKAN's fallback download mechanism to download files, so whatever storage mechanism is behind it is abstracted away. Same goes for upload - ckanext-blob-storage has no knowledge or dependency on Azure.
A script to migrate resources can be written based on the CKAN API (+ Python SDK, as it will need access to Git LFS as well as CKAN), or by directly accessing the DB and Azure Blob Storage. The former is most likely preferrable and easier though the latter may be faster.
Acceptance
Tasks
lfs_prefix
andsha256
attributes setlfs_prefix
andsha256
attributesThis can easily be parallelized if we need to (e.g. if we need to run fast if the system is taken down) or run slowly in the background. It can also be restarted if needed and will continue from where it stopped.