OregonDigital / OD2

Next generation of Oregon Digital ( https://oregondigital.org ) digital collections platform, built on Samvera Hyrax ( https://github.com/samvera/hyrax/ )
18 stars 1 forks source link

OD2 set up server for import files #2958

Open lsat12357 opened 1 year ago

lsat12357 commented 1 year ago

Descriptive summary

Set up designated server space to be used for uploading content files to be imported into OD2. Somewhat analogous to /data1/batch. In fact could be /data1/batch repurposed. wickr suggests that files would not need to remain in the directory for longer than 3 months. It is also possible that the current archives sitting on /data1/batch can be removed? as all of that content should be migrated and now duplicated on /od2_migration.

Expected behavior

Designated curators with admin access? would upload (via scp? kubectl cp?) compressed batches to the server. In order to use browse_everything it looks like the curator would need to follow up with untarring/unzipping the package. For use with the upload zip option, the package can remain zipped prior to start of import. It is possible that bulkrax can be configured to unzip in this space as well, unclear if this would be desirable. Currently it is using tmp/shared/imports, and side note, it looks like we'll need to figure out if we can clean these up after the import is complete?

Related work

once we have the path, we can fix the browse_everything config:

2912

Accessibility Concerns

Add any information here to indicate any known or suspected accessibility issues for this ticket

decimalator commented 11 months ago

/data/tmp/shared/bulk has been created for this purpose