search

buda-base / drs-deposit

Harvard DRS Deposit base
1 stars 0 forks source link

Wrong files in built batches #66

Open jimk-bdrc opened 6 years ago

jimk-bdrc commented 6 years ago

Some 500 batches, comprising 4500 objects, have been mis-built since 18 June. These were all files which had multiple volumes. The bug was such that the files from the first volume were copied into the subsequent volumes, but given the subsequent volume's OSN, replicating the first volume's content in each volume.

The list of bad batches is: UnDepositedBadBuildPaths.txt

They were found by looking at all the batchbuilds whose paths were in /Volumes/DRS_Staging/prod/batcBuilds which had more than one volume. The SQL queries generating this are in FindBatBatchSql

jimk-bdrc commented 6 years ago

Fixed running the SQL script in 340aa21. Testing going ahead.

jimk-bdrc commented 6 years ago

turns out I only found half the objects. I did a more canonical scan using Python util scanBuild and discovered more. I've removed them from $PR/batchBuilds (they're in $PR/badBatchBuilds) The list of newly found bad objects are in More bad June 18 Bad Objects Built. See 743d0a0. Needs to have Vitaly's guy delete them when he returns next week.

jimk-bdrc commented 3 years ago

Fixed in dawn of time