Bulkrax x Embargos/Leases

[ShanaLMoore]

Summary

SOW

SoftServ will expand Bulkrax functionality by allowing the input of Embargo (and lease?) rules via CSV. Fields will include:

• Restricted To - this text field defines the visibility level during Embargo
• Restricted Until - this date field defines the end date of the Embargo
• Open To After Restriction - this text field defines the visibility level upon expiration of the Embargo

Visibility levels will correspond with current levels in Hyku Commons: Public, Institutional, Private. Date inputs will follow a single standard format; functionality will not be created to convert dates or handle multiple formats. This work will be contributed back to Bulkrax and Hyku by SoftServ with acknowledgement to Hyku for Consortia and the University of Pittsburgh.

Acceptance Criteria

• [ ] As an admin user I can upload Embargo rules via a CSV through Bulkrax
• [ ] CSV fields for Embargoes include:
  • [ ] Restricted To - this text field defines the visibility level during Embargo
  • [ ] Restricted Until - this date field defines the end date of the Embargo
  • [ ] Open To After Restriction - this text field defines the visibility level upon expiration of the Embargo
• [ ] This has been contributed back to Bulkrax and Hyku

Screenshots or Video

Testing Instructions

Notes

Embargo format idea (per team meeting)

pitt-embargo.csv

model,title,creator,rights_statement,bulkrax_identifier,visibility_during_embargo,embargo_release_date,visibility_after_embargo
Image,An Image,KKW,http://rightsstatements.org/vocab/InC/1.0,123456789,restricted,2024-04-16,open

Lease methods/options

work.visibility
work.visibility=
work.visibility_after_embargo
work.visibility_after_embargo=
work.visibility_after_lease
work.visibility_after_lease=
work.visibility_changed?
work.visibility_during_embargo
work.visibility_during_embargo=
work.visibility_during_lease
work.visibility_during_lease=
work.lease_expiration_date
work.lease_expiration_date=
work.lease_history
work.lease_history=
work.lease_id
work.lease_id=
work.lease_indexer_class
work.lease_visibility!

so header proposal: visibility_during_lease lease_expiration_date and visibility_after_lease

Edge case

Validations should be considered for the following scenarios:

• [ ]

[ShanaLMoore]

Embargo

Manually created work:

Lease

Manually created work:

[ShanaLMoore]

Current bulkrax options from the parser:

Is adding embargo/lease to the importer field part of this scope of work ? ANSWER: No, this is not in scope. We will focus on the CSV only, per Rob.

~If embargo or lease is added to this view, additional fields should render so that the user can set the dates.~

Visibility via csv options:

    # The visibility of the record.  Acceptable values are: "open", "embaro", "lease", "authenticated", "restricted".  The default is "open"
    #
    # @return [String]
    # @see https://github.com/samvera/hydra-head/blob/main/hydra-access-controls/app/models/concerns/hydra/access_controls/access_right.rb Hydra::AccessControls::AccessRight for details on the range of values.
    # @see https://github.com/samvera/hyrax/blob/bd2bcffc33e183904be2c175367648815f25bc2b/app/services/hyrax/visibility_intention.rb Hyrax::VisibilityIntention for how we process the visibility.
    def visibility
      @visibility ||= self.parser_fields['visibility'] || 'open'
    end

[ShanaLMoore]

Rails console inspection of manually created works:

Lease object got created

[ShanaLMoore]

TODO: test if it just works by adding to the CSV: "lease_expiration_date_dtsi":"2024-04-17T00:00:00Z", "visibility_after_lease_ssim":["restricted"], "visibility_during_lease_ssim":["open"],

[ShanaLMoore]

Blocked merging it in for PITT until pals knapsacking is done.

[ShanaLMoore]

TODO:

• [x] update hyku (https://github.com/samvera/hyku/pull/2191)
• [ ] update pals knapsack before deploying to staging

[ShanaLMoore]

Confirming that embargo/lease bulkrax works for active fedora and valkyrie

[jillpe]

Will test it once the IIIF splitting fix is merged

[ShanaLMoore]

QA RESULTS: Pass ✅

Tested on: Staging

• [ ] As an admin user I can upload Embargo rules via a CSV through Bulkrax
• [ ] CSV fields for Embargoes include:
  • [ ] Restricted To - this text field defines the visibility level during Embargo
  • [ ] Restricted Until - this date field defines the end date of the Embargo
  • [ ] Open To After Restriction - this text field defines the visibility level upon expiration of the Embargo

[ShanaLMoore]

QA RESULTS: PASS ✅

Tested On: STAGING

Sample: embargo-lease.csv

[ShanaLMoore]

sample file for split:

split-embargo-pals.zip

[ShanaLMoore]

New Issue since updating submodule: https://demo.palni-palci-staging.notch8.cloud/importers/7/entries/8?locale=en

[jillpe]

Blocked: There are failing specs (https://github.com/samvera/hyku/pull/2260) and we need to update the submodule

[jillpe]

SoftServ QA: ✅ File used for import: split-embargo-pals.zip Importer

https://github.com/user-attachments/assets/637d1978-e3b0-4d31-8071-5442e8cd19de

[ctgraham]

I tried submitting two imports: https://demo.palni-palci-staging.notch8.cloud/importers/28?locale=en and https://demo.palni-palci-staging.notch8.cloud/importers/30?locale=en

Both listed a "Total Works:" count as expected, but neither lists any Works, and neither presents any error messages.

[ShanaLMoore]

TODO: Look into why jobs don't appear to be running in staging

[ShanaLMoore]

@ctgraham

For your uploads, I am seeing this error in the logs.

I agree that not displaying the error is problematic (we can create a separate ticket for error handling), but could you fix the spreadsheets and try again?

[["status_message", "Failed"], ["error_class", "StandardError"], ["error_message", "failed_validation - Resource type can't be blank"]

[ShanaLMoore]

Handing this back to cc @laritakr to rework the combined_files logic.

STEPS TO REPRODUCE

• Locally, run git update submodule --remote so that pals gets the latest bulkrax code (which contains bug fixes they'll need).
• Download the original file from clinton's importer
• Make sure required fields are there, like type and keyword. (I also updated headers: degree_name => degree and degree_level => level per pals' bulkrax field mappings).
  • the client mentioned issues with using the proper field mappings so I wanted to test this too.
• at this point if you upload you'll get a FILE NOT FOUND ERROR:
  • Bulkrax does NOT support files with spaces so I added underscores to the filename in the item column of the csv. I also renamed the file in the files directory of the zip to include underscores.
  • rezip the file and import it Error:

    [["status_message", "Failed"], ["error_class", "NoMethodError"], ["error_message", "undefined method `[]' for nil:NilClass"], ["error_backtrace", "---\n- \"/usr/local/bundle/bundler/gems/bulkrax-cd9b58226ef2/app/factories/bulkrax/valkyrie_object_factory.rb:285:in\n

The problem is combined_files is an empty array for this scenario. There are no remote files or thumbnail urls.

However, FOR TESTING PURPOSES ONLY the attached sample file contains both columns (item and remote_files), to inspect combined_files and uploaded_files at the following debug location.

remote-and-local-files.zip

[ShanaLMoore]

TODO: set up pairing time with Rob to figure out blocker

[laritakr]

TODO: Update Bulkrax to released version and retest