bcgov / ckanext-bcgov

BC Data Catalogue source code, main ckan extension
http://catalogue.data.gov.bc.ca
GNU Affero General Public License v3.0
24 stars 23 forks source link

Update all records that contain resources which are missing the resource_storage_location field. #635

Closed dkelsey closed 5 years ago

dkelsey commented 5 years ago

TASK

since the 1.7 release resource_storage_location has been a mandatory field. Records the pre-existed the release did not contain a value for this field (*)

* some actually did. Why? How?

Some records have more than one resource which omits this field. With that condition, the user interface will not allow editors to update their record:

The task is:

Write a one-off script to add/set the resource_storage_location field for resources where this field is missing, and update all records that contain these resources.

dkelsey commented 5 years ago

I ran my script after some analysis and testing. I backed-up each record to a JSON file before making the update.

Every resource in every metadata record now has a resource_storage_location field.

My analysis was to see how difficult it would be to set the values correctly. The current breakdown of resource_storage_location:

Label Count
External 2694
BCGW Data Store 2343
Catalogue Data Store 1179
EDC Data Store 769
BCGW DataStore 434
BC Geographic Warehouse 260
Web or FTP Site 191
Pub.data.gov.bc.ca 147
Ministry or Other Database 21
Esri ArcGIS Onlin 20
File System 8
Unspecified 6

Note: