change type for locations with PAA = DARE Precision 2: 3 pts

[paregorios]

As a Pleiades admin with CLI access on production, I can run a data modification script that changes the Location Type field values on any published Pleiades Location object to the single value "associated modern" if the published Location object already references the "DARE Precision 2" Positional Accuracy Assessment and does not already have "associated modern" in its multivalue location type field. If, "associated modern" does already appear in location type, then the Location object is skipped.

The script must:

• Have a "dry-run" option that performs all actions except for committing the changes to the database, leaving the data unchanged.
• Report back to the command line or a data file:
  • the paths of all Locations changed and their original location type value(s)
  • the paths of all matching Locations that were found to already have the type value "associated modern"
  • any errors encountered by the script together with the corresponding object path
• Handle large numbers of changes in a single run (by batching database transaction processing?)
• Ensure all modified objects are properly reindexed, if needed.

For example:

• This DARE Location references PAA DARE Precision 2 and has a location type value of "representative". The script should replace the existing value(s) in the location type field, and add only the value "associated modern". The path "/places/383624/dare-location" will be added to the script output under the "changed" category, together with the previous location type value(s), in this case ["representative",]
• DARE Location of Modern Le Candéou also references PAA DARE Precision 2, but its location type already has the value "associated modern" so no change is required. The path "/places/157833/dare-location-of-modern-le-candeou" will be added to the script output under the "unchanged" category.

[chiruzzimarco]

@paregorios The PR got deployed to staging and can be tested.

Here is an example output from a local dry run:

$ bin/instance run scripts/update_dare2_locations_type.py --dry-run

Dry Run: True
..............................................................................................................................................................................................................................................................................................................................................................................................................................................Done!

UPDATES: 160 records were updated successfully!
places/334485/dare-location: '(u'representative',)' ==> '(u'associated modern',)'
places/413144/dare-location: '(u'associated_modern',)' ==> '(u'associated modern',)'
places/422927/dare-location: '(u'representative', u'associated_modern')' ==> '(u'associated modern',)'
places/109349/dare-location: '['representative', '']' ==> '(u'associated modern',)'

....

REDUNDANT: 319 records were skipped because changes were already applied.
places/157833/dare-location-of-modern-le-candeou
places/609484/dare-location
places/629075/dare-location

...

PROBLEMS: 1 records were skipped due to problems:
places/135066932/old-palace: 'str' object has no attribute 'id'

🙅 NO CHANGES APPLIED to the database (--dry-run selected)

If i run the script without the --dry-run option, i get some errors and warnings when committing changes to the database:

WARNING pleiades.vaytrou Failed to index_doc 1069002108: [Errno 111] Connection refused

...

ERROR Zope.UnIndex KeywordIndex: unindex_object could not remove documentId 1069002108 from index object_provides.  This should not happen.
Traceback (most recent call last):
  File "/home/marco/Desktop/projects/pleiades3-buildout/eggs/Products.ZCatalog-3.0.2-py2.7-linux-x86_64.egg/Products/PluginIndexes/common/UnIndex.py", line 157, in removeForwardIndexEntry
    indexRow.remove(documentId)
KeyError: 1069002108

Changes gets applied regardless, and i'm not sure if this is a problem of my local dev environment or a real problem that needs to be addressed.

[alecpm]

@chiruzzimarco I think it's a local dev issue. There's another application ("vaytrou") you need to have running for geospatial indexing to work.