search

denshoproject / ddr-local

Web UI used for interacting with DDR collections and entities on a local machine.
Other
3 stars 0 forks source link

Batch import of new files in web UI #317

Closed gjost closed 11 months ago

gjost commented 1 year ago

ddrimport files has pretty much worked for awhile now. Batch export/import was always intended to be a feature usable through the web UI (the "import csv (objects,files)" links in the footer). Let's do this!

Do a minimal implementation and see how it goes.

gjost commented 1 year ago

12:03 sara.beckman - Tested the ddrimport in the Editor UI on 10.0.1.137. I imported mezz, master, and transcript files to ddr-testing-40410. I was able to successfully import the files. The log files were created in a subfolder. I also tested the CSV validation -- it dedicated a mismatch between file name & basename_orig. I also deleted a required header in the csv and the CSV validator caught it.

12:08 sara.beckman - One thing I noticed - another test I ran as a sort of sanity test. I tried a file import before I imported entities just to make sure it wouldn't work. It didn't and gave a clear Failure notice in celery. It gave a log file with the error output. However that log wasn't saved to the log folder. Could that be saved like the success log does?

12:09 gjost - Was the error on the batch file import (sounded like that worked before) or the batch entity import, which i haven't worked on yet?

12:10 sara.beckman - file import in the editor 12:11 - I just wanted to see what would happen if an archivist tried to import the files when the entities weren't present. Basically trying to get a failure notice in celery 12:12 - The failure notice was clear and had a link to the log file. Which clearly stated that the entities weren't in the repo. The log file just didn't save in the ddrshared folder like the success log files. 12:13 - I think saving even failure log files to the ddrshared folder would be beneficial

12:17 gjost - This case probably should have failed in the validation stage 12:22 sara.beckman - Adding it to the validator when you select the CSV would work too. But the import failing right away with the log is pretty clear and works too -- but catching it first would probably cause the archivists less distress

TODO Try to import files for non-existent entities

gjost commented 1 year ago

Work completed as of ddr-local commit e22f4b3328 and merged in to master.