search

denshoproject / ddr-local

Web UI used for interacting with DDR collections and entities on a local machine.
Other
3 stars 0 forks source link

ddrimport: Check for already-existing files, return better error messages on import fail #339

Open gjost opened 9 months ago

gjost commented 9 months ago

User attempted to re-import a file to an Entity that already had that file attached with the same role. The checker did not catch this, the import was attempted and failed with an uninformative error message. We need the checker to catch this case and print a better error message.

2023-12-18 06:20:44,392630 INFO     Running checks on /media/sf_ddrshared/329/Masters/Domoto6_ddr-densho-329_Masters_MetadataFilesforImport - MetadataFilesforImport.csv

2023-12-18 06:20:49,945400 INFO     ========================================================================
2023-12-18 06:20:49,947446 INFO     BEGIN BATCH FILE IMPORT
2023-12-18 06:20:49,949342 INFO     ========================================================================
2023-12-18 06:20:50,375246 INFO     batch import files ----------------------------
2023-12-18 06:20:50,378454 DEBUG    csv_dir='/media/sf_ddrshared/329/Masters'
2023-12-18 06:20:50,380370 DEBUG    entity_class=<class 'DDR.models.entity.Entity'>
2023-12-18 06:20:50,382226 DEBUG    <git.repo.base.Repo '/var/www/media/ddr/ddr-densho-329/.git'>
2023-12-18 06:20:50,384008 INFO     333 rows
2023-12-18 06:20:50,385798 INFO     csv_load rowds
2023-12-18 06:20:50,728195 INFO     - - - - - - - - - - - - - - - - - - - - - - - -
2023-12-18 06:20:50,730230 INFO     Updating existing files
2023-12-18 06:20:50,732091 DEBUG    0 updated in 0:00:00.000009
2023-12-18 06:20:50,733859 INFO     - - - - - - - - - - - - - - - - - - - - - - - -
2023-12-18 06:20:50,735565 INFO     Adding new files
2023-12-18 06:20:50,737161 INFO     addfile log to /media/sf_ddrshared/329/Masters/log/Domoto6_ddr-densho-329_Masters_MetadataFilesforImport - MetadataFilesforImport.log
2023-12-18 06:20:50,738851 INFO     + 1/333 - ddr-densho-329-716 (ddr-densho-329-716-1_master.tif)
2023-12-18 06:20:50,740383 DEBUG    | parent <DDR.models.entity.Entity entity:ddr-densho-329-716>
2023-12-18 06:20:50,742402 INFO     ------------------------------------------------------------------------
2023-12-18 06:20:50,743893 INFO     DDR.models.Entity.add_local_file: START
2023-12-18 06:20:50,745570 INFO     rowd: {'id': 'ddr-densho-329-716', 'external': False, 'basename_orig': 'ddr-densho-329-716-1_master.tif', 'role': 'master', 'sort': '1', 'label': 'photo left', 'rights': 'cc', 'public': '0', 'mimetype': 'image/tiff', 'external_urls': [], 'links': '', 'tech_notes': 'VueScan 9.5.81; Epson Expression 12000XL', 'digitize_person': 'Merryman, Micah', 'identifier': <DDR.identifier.Identifier entity:ddr-densho-329-716>, 'path_abs': PosixPath('/media/sf_ddrshared/329/Masters/ddr-densho-329-716-1_master.tif')}
2023-12-18 06:20:50,747323 INFO     parent: ddr-densho-329-716
2023-12-18 06:20:50,749031 INFO     src_path=PosixPath('/media/sf_ddrshared/329/Masters/ddr-densho-329-716-1_master.tif')
2023-12-18 06:20:50,751270 DEBUG    actions {'label': 'new-internal', 'attrs': 'calculate', 'ingest': 1, 'rename': 0, 'access': 1}
2023-12-18 06:20:50,752925 DEBUG    Actions: attrs calculate
2023-12-18 06:20:50,754667 DEBUG    Examining source file
2023-12-18 06:20:50,756435 DEBUG    | check dir /media/sf_ddrshared/329/Masters/ddr-densho-329-716-1_master.tif (| src_path)
2023-12-18 06:20:50,759652 DEBUG    | file size 473451968
2023-12-18 06:20:50,761621 DEBUG    | hashing
2023-12-18 06:20:51,673853 DEBUG    | md5: e2223054b7d7e6978ee7125cd4468a5a
2023-12-18 06:20:52,442544 DEBUG    | sha1: 58f57c99661892094469f0a1724fc0f2a4bf9afa
2023-12-18 06:20:53,833396 DEBUG    | sha256: e3cf21b6b2a4577a5757e6c2ea987ded276dfde4995beaa5bca960a421a06072
2023-12-18 06:20:53,835254 DEBUG    | md5 e2223054b7d7e6978ee7125cd4468a5a
2023-12-18 06:20:53,836946 DEBUG    | sha1 58f57c99661892094469f0a1724fc0f2a4bf9afa
2023-12-18 06:20:53,838563 DEBUG    | sha256 e3cf21b6b2a4577a5757e6c2ea987ded276dfde4995beaa5bca960a421a06072
2023-12-18 06:20:53,840225 DEBUG    | extracting XMP data
2023-12-18 06:20:53,842426 DEBUG    Identifier
2023-12-18 06:20:53,844133 DEBUG    | idparts OrderedDict([('model', 'file'), ('repo', 'ddr'), ('org', 'densho'), ('cid', '329'), ('eid', '716'), ('role', 'master'), ('sha1', '58f57c9966')])
2023-12-18 06:20:53,845951 DEBUG    | identifier <DDR.identifier.Identifier file:ddr-densho-329-716-master-58f57c9966>
2023-12-18 06:20:53,847623 DEBUG    File object
2023-12-18 06:20:53,849865 DEBUG    | file_=<DDR.models.files.File file:ddr-densho-329-716-master-58f57c9966>
2023-12-18 06:20:53,851882 DEBUG    | file_.basename_orig='ddr-densho-329-716-1_master.tif'
2023-12-18 06:20:53,853718 DEBUG    | file_.path_abs='/var/www/media/ddr/ddr-densho-329/files/ddr-densho-329-716/files/ddr-densho-329-716-master-58f57c9966.tif'
2023-12-18 06:20:53,855531 DEBUG    | file_.mimetype=''
2023-12-18 06:20:53,857334 DEBUG    | file_.size:   473451968
2023-12-18 06:20:53,859036 DEBUG    | file_.sha1:   58f57c99661892094469f0a1724fc0f2a4bf9afa
2023-12-18 06:20:53,860633 DEBUG    | file_.md5:    e2223054b7d7e6978ee7125cd4468a5a
2023-12-18 06:20:53,862310 DEBUG    | file_.sha256: e3cf21b6b2a4577a5757e6c2ea987ded276dfde4995beaa5bca960a421a06072
2023-12-18 06:20:53,864017 DEBUG    Actions: rename 0
2023-12-18 06:20:53,865629 DEBUG    Actions: ingest 1
2023-12-18 06:20:53,867255 DEBUG    Copying
2023-12-18 06:20:53,868790 DEBUG    | cp /media/sf_ddrshared/329/Masters/ddr-densho-329-716-1_master.tif /var/www/media/ddr/ddr-densho-329/files/ddr-densho-329-716/files/ddr-densho-329-716-master-58f57c9966.tif
2023-12-18 06:20:53,877309 ERROR    Import failed -- rolling back