IMPORT: Ingest files

[shanitajohn]

ACCEPTANCE Given that each object (or line) in the CSV has a primary file, and zero or more secondary files and that files are accessible from the server

• [x] I can start a CSV import from the command line
• [x] The primary file for each CSV import object becomes the representative file in the work ~- [ ] Secondary files are also attached to the work~
• [x] File characterization and derivative generation is run (save goes through Hyrax actor stack)

NOTES

• All ETDs have at least one file
• All ETDs have a least one PDF which is primary
• Some ETDs have additional files attached

[no-reply]

I can start a CSV import from the command line

is dealt with by #243.

[no-reply]

Secondary files are also attached to the work

Do we have an example of a bepress export with a secondary file? The existing demo export we have doesn't include this, as far as I can tell.

cc: @mark-dce @banerjek

[no-reply]

The middle check-box in this is blocked by some confirmation of what secondary files will look like.

The other remaining items are:

• [x] Set up a real User to be the creator for imported objects;
• [x] Make the directory containing files for import configurable.

[mark-dce]

For the ETD specific use case, see the items listed under "Additional Files" on https://digitalcommons.ohsu.edu/etd/3725/ - this is specifically in scope.

For future scope, see the "PDF" and "Data Transcription" on https://digitalcommons.ohsu.edu/public-health-data/349/ - this is future scope, but our current work would ideally be extensible to handle this use case fairly easily in the future either by OHSU or DCE.

If we need additional secondary file examples, probably contact Kyle directly.

Hi Shanita,

Here is an example of a PDF with files attached:

http://digitalcommons.ohsu.edu/etd/3725

In a totally separate collection, a data file is attached to the vast majority of PDFs, see http://digitalcommons.ohsu.edu/public-health-data/349/

From what I can tell, we have just shy of 200 documents with at least one data file attached, maybe a couple dozen tops that have more than one file attached.

This means it would be desirable to have an automatic process to migrate attached data files – but that it would also be very acceptable to manually deal with the records that have multiple files attached .

[no-reply]

With the secondary file requirement removed (BePress does not export the filenames), this ticket is now complete and ready to close.