Script to set number of intended children for all Assets

[bkiahstroud]

Story

PR

• https://github.com/WGBH-MLA/ams/pull/732

Ref

• https://github.com/scientist-softserv/dev-ops/issues/729

A "missing child" validation was added for Assets in https://github.com/scientist-softserv/dev-ops/issues/729. With the changes made for that ticket, all future XML imports/ingests will have each Asset count its number of intended children and save that number to be used later for validation. The purpose of this ticket is to do the same, except for existing Asset records.

A script needs to be written that will loop through all existing Assets and set both the :intended_children_count and :validation_status_for_aapb properties.

Acceptance Criteria

• [x] Write a script that will sets the values of :intended_children_count and :validation_status_for_aapb for all existing Asset records
• [ ] Test on demo
• [ ] Run the script on production

Testing Instructions

1. After the script has been run, open a rails console
2. Run ActiveFedora::SolrService.get('-intended_children_count_isi:[* TO *] has_model_ssim:Asset')['response']['numFound']. Verify the result is 0
3. Run ActiveFedora::SolrService.get('-validation_status_for_aapb_tesim:[* TO *] has_model_ssim:Asset')['response']['numFound']. Verify the result is 0
4. find a random Asset that has at least one child record
5. Check the value of :intended_children_count. Make sure it is not nil or 0
6. Check the value of :validation_status_for_aapb. Make sure it is not nil
7. Repeat steps 4-6 five times

[ShanaLMoore]

Assigning @laritakr to this for follow up support per slack convo.

[bkiahstroud]

Needs Rework

• #raw_data_from_batch_item needs to account for when batch_item.source_location is nil
  • batch_item.source_data should be available in this case. Parse the data and count the number of intended children
• #raw_data_from_batch_item needs to account for when batch_item.source_location is present but the file cannot be found on the file system
  • Fall back on the S3 bucket

[orangewolf]

this is blocked because I believe it will be done due to the resaving of every record, but we need to verify that.

[orangewolf]

@bkiahstroud now that we've been through most of the fedora -> pg migration, can you confirm if this still needs to be run or was it updated when we resaved the records?

[bkiahstroud]

@bkiahstroud now that we've been through most of the fedora -> pg migration, can you confirm if this still needs to be run or was it updated when we resaved the records?

@orangewolf assuming we're only concerned with Valkyrie records, I don't believe this needs to be run anymore.

# Count: Valkyrie Asset SolrDocuments that are missing a :intended_children_count_isi value
ActiveFedora::SolrService
  .get('has_model_ssim:Asset AND valkyrie_bsi:true AND -intended_children_count_isi:[* TO *]')
  .dig('response', 'numFound')
=> 0

# Count: Valkyrie Asset SolrDocuments that are missing a :validation_status_for_aapb_tesim value
ActiveFedora::SolrService
  .get('has_model_ssim:Asset AND valkyrie_bsi:true AND -validation_status_for_aapb_tesim:[* TO *]')
  .dig('response', 'numFound')
=> 0

[bkiahstroud]

@jillpe what's left to do here?