"Validation" as a workflow step in between Import and Transfer

[jblowe]

We've discussed how to improve the workflow for various use cases (the current workflow is geared towards initial migrations of data into CSpace).

Were we to insert a new step into the process, we'd have to refactor the converter to handle it. Reading the code, I am guessing that this would entail:

• Making a new "job", let' call it Validation. It would look very much like Import_job or Transfer_job, but the lookup bits in Import job would be moved into Validate_job, etc.
• Making a new batch (or something) in the mongodb for the validated data, and having Transfer_job refer to this batch.
• Exposing this in the UI

Is this the way to go about this?

[mark-cooper]

@jblowe yes, broadly I think it is but I think we may want to change the workflow (and potentially the data model) a bit.

UI -> create batch. A batch has some metadata and a file.

When viewing a batch it now has steps:

All batch

• import (load all the csv rows, create DataObjects) -- or this could continue to happen on upload?

All batch or individual selections, none of these things happen automatically on upload

• validate (checks DataObject is valid -- we need to figure out how rules will be expressed)*
• generate (create CspaceObject [metadata and xml record]*
• transfer [create, update, delete]

Each "step" requires the previous one to complete successfully, so you cannot transfer without first validating then generating.

• These have the potential to change quite a bit in terms of they're integrated as the mapper gem matures: https://github.com/collectionspace/collectionspace-mapper