search

harvard-lil / perma

Indelible links
413 stars 70 forks source link

When IA uploads "fail" #2963

Open rebeccacremona opened 3 years ago

rebeccacremona commented 3 years ago

I noticed a lot of "was already uploaded to IA: skipping" going by in the logs... That happens if the item already exists in IA: https://github.com/harvard-lil/perma/blob/396953c14e8b7bdb8e01df34324ee93b2c994534/perma_web/perma/tasks.py#L1640. It looks like a number of these records are fine in IA, but our DB has the upload attempt marked as failed. See, for instance, 4F3Y-F5XT.

Let's get a list of all "failed" uploads and check on them.

rebeccacremona commented 3 years ago

deletions too Link 27E5-FHSH not present in IA: skipping.

rebeccacremona commented 3 years ago

I found 291 'failures' that were fine, updated to 'completed':

['2776-FRMP', '2GXD-EB3K', '2QWT-SUYT', '2SZE-M6GF', '2U5D-UF94', '2W45-NDYT', '2WA9-47WK', '2WAA-JHYH', '2XR4-EZXE', '2YZB-JUFX', '2Z4Z-WRBX', '2Z64-S6HU', '328D-ECJ8', '33TF-THA6', '38LW-7HSN', '3HFQ-MPPE', '3MUT-USEA', '3MUW-NGU9', '3MXP-JJ5T', '3N3Z-E7CR', '3N6E-359Y', '3P6S-W736', '3RJ6-J7NL', '3S2R-SD7E', '3WN2-UWED', '3Z8P-YEGU', '3ZDD-QJYU', '43TT-WX7K', '467X-ZS64', '468H-K92G', '48UC-6G89', '48V2-MAJB', '48V9-FVB5', '48XK-MBVL', '48XM-BBZ2', '48XX-NST3', '48Y5-JHLL', '48YJ-AX5S', '494Y-YGF5', '495S-ZKZB', '49CL-R7KK', '49EW-RH2T', '49FF-YB9M', '49G2-TWFT', '49H3-NLDK', '49JE-NCNE', '49JN-R8K5', '49PG-WX76', '49SL-R3E5', '4A5D-2G6K', '4AHC-FP4X', '4AQH-HE5Q', '4CYE-TCY8', '4DGT-H6ZS', '4DLZ-UCD6', '4DMS-YFSC', '4DRX-R9E2', '4DST-C2ZD', '4DZ5-XTSN', '4E3W-CJVX', '4E4J-9FPA', '4E4R-FX6D', '4E6R-53H9', '4E8D-GZUC', '4E9G-TRMN', '4E9L-JJDQ', '4EAF-9Q7R', '4EUX-PX9L', '4EZE-GE6U', '4F3Y-F5XT', '4F62-385Q', '4F6S-KCBE', '4M4S-MESH', '4MAD-KMVA', '4MNN-TVV3', '4NF7-PBUM', '4NFF-2LM5', '4UAF-ZYUP', '4W3C-BY88', '4YWP-4M96', '4Z5V-23XZ', '54SB-L9SW', '553V-W3KZ', '58FW-WEMX', '58KG-7RUE', '58LR-GHE5', '58MK-DKN9', '58MV-M5LU', '58Z4-SFXG', '58Z8-KP8P', '58ZJ-GQJR', '593E-Z3QE', '593T-WJ4W', '597H-K8WU', '597U-9M44', '598K-XM2U', '598N-M9PK', '599B-PV35', '59CG-Q549', '59ES-ZJ69', '59FX-VBD5', '59GJ-RKM4', '59GT-JT9H', '59LN-NEZR', '5ENX-ZZA9', '5GU8-3TGU', '5KYS-EALZ', '5L8X-2RBA', '5M3V-RKP8', '5N66-3MSX', '5SDR-C5NK', '5SFS-XMGA', '62VS-QUBQ', '65CC-HQ5C', '662T-YF75', '6FA3-4DML', '6FBY-C3N5', '6M5K-QAWM', '6XDY-H9BE', '829M-H8UQ', '8H4J-GCHP', '8QHK-BUQD', '8SU7-ANUT', '8TGE-B5XD', '8VDQ-2Z2U', '9A48-VYMP', '9HA5-LYTC', '9YH5-ZJHR', 'A3CF-ZZ9B', 'B3WL-EZ28', 'B5AY-JU9Z', 'BT4J-S862', 'C2XL-KRU2', 'D9F9-GEQA', 'DCG7-2CWM', 'DM3D-E8EJ', 'DMK7-22RV', 'DY2A-Y456', 'E3VR-T9EZ', 'EW9W-PFFX', 'EWQ2-W5XD', 'EZ96-XWV8', 'F2AR-JWEL', 'F3JZ-HZ7C', 'F3SE-BX6Z', 'F9WM-47XR', 'FYD6-TK59', 'FYY6-WW8C', 'G38Z-F9LL', 'G4LX-GE28', 'GC7B-SUHE', 'GC7R-TVQV', 'GC85-7V3K', 'GC8B-Z76S', 'GE7K-FEVV', 'H2KV-ZES9', 'H4EQ-VCXV', 'H85W-XHK7', 'H8G7-8W9N', 'H9FC-ZYY8', 'HEE5-YXRW', 'HEV4-H2AE', 'HK34-FRWM', 'HQ2L-7WVH', 'HX25-B8NG', 'HX2B-3V55', 'J385-E94A', 'J39N-CAKC', 'J7Q2-5QQ8', 'J7ZJ-4PBB', 'J84N-GK8V', 'J9FQ-NVBD', 'JE88-9N3Q', 'JR4P-MBMF', 'JS7W-MMXS', 'JSR7-RBP6', 'K2P5-V64B', 'K2U3-MY4W', 'K32W-N7CX', 'K6B7-PA7K', 'K85J-USL2', 'K85J-X8J7', 'K8AJ-EU2W', 'K8AK-8Q3Q', 'K8R8-7AVC', 'K9EF-2MMZ', 'K9EQ-EX3J', 'K9EV-FXZH', 'K9VP-32XG', 'KC8L-WULC', 'KHS8-B29Q', 'KKR6-R9Y5', 'KKS9-NKXC', 'KL53-JQQY', 'KN6V-3QCV', 'KP5S-B5TK', 'KP5Y-6CH7', 'KP5Z-9FZ8', 'KP68-PUWK', 'KP6G-LEX9', 'KP6M-U7M7', 'KQ4K-GUGT', 'KS3P-2XHX', 'KV8R-7TK9', 'L4JH-6VT2', 'L92P-LHZH', 'L9RE-KJPK', 'LU59-FF83', 'LYC6-GW7Q', 'M65C-DLY5', 'M96P-DZPC', 'M9A6-QNXG', 'M9MH-Z462', 'MBJ5-H69D', 'MC8Z-FR5Z', 'MGG7-TKKS', 'MHV2-2RFQ', 'ML4E-8UC5', 'N7D4-JT9A', 'NEQ5-JQGF', 'NS6T-N88C', 'NV3L-92Y7', 'PE9Q-LVMG', 'PTE3-Q7PJ', 'PW96-KKGB', 'Q55K-5NEJ', 'QY8H-2B4E', 'QZ5V-GE2N', 'R2DD-WJMS', 'R5UY-YCGH', 'R6P2-UMXT', 'RAQ5-36XP', 'RD2Y-VPUH', 'SCE4-U4KU', 'SF3L-U8FL', 'SN92-5PD4', 'SQ77-4M3D', 'SY9R-GUGN', 'SZ23-D23T', 'T2MX-N4HF', 'T3M7-6AE9', 'T4BY-3D4Y', 'T58K-U9N3', 'T6H6-2PJY', 'TK7R-U4XX', 'TK7S-8PGW', 'TN8X-A7YY', 'TP4K-HQGP', 'TP4M-3D7E', 'TP4N-H36J', 'TP4P-W4QU', 'TP4R-CN56', 'TS9F-HHZ9', 'TX3U-V8XZ', 'U2V8-52YU', 'U5BL-KKSG', 'U743-XX53', 'UZW7-L4KU', 'V3MC-JNNY', 'V3MQ-PT6S', 'V4D8-9S57', 'V69G-RBG9', 'V7Q2-B7WR', 'VWF8-SLMG', 'W25E-22BL', 'W4K2-844N', 'W59T-DPHC', 'W6XC-93SM', 'W95Y-63SP', 'WFL4-QTCE', 'WG99-D7N2', 'WL2D-ABGE', 'WMZ2-LWLP', 'WP6A-M54L', 'WPT7-P933', 'WV48-D243', 'WVJ3-Q9QY', 'X8CL-XQHZ', 'XFF6-AKWJ', 'XW2S-DRCS', 'XZ7U-LGWG', 'YNU6-M23T', 'Z3QU-J2B2', 'Z5L2-JKJU', 'Z823-6WSR', 'ZA59-KJH4', 'ZD76-5MLK', 'ZEM3-HDCQ', 'ZLE9-K8QV', 'ZV3C-4J6Z', 'ZX92-XP4Y']
rebeccacremona commented 3 years ago

I found 5 that were not uploaded. Doing so synchronously in the shell.

372N-88RU, 37AE-YGXU, 3U4B-C8CQ, 5289-5A86, G4XV-ZP7S
rebeccacremona commented 3 years ago

I found 30 that were only partially uploaded. I'm going to delete then re-upload.

'3CUK-WX8J', '46P4-MWA2', '68VA-TJQL', '7BCV-GSSJ', '7C29-4NU8', '7QBJ-WEGR', '9Q55-T94M', 'AQP8-27LW', 'DZH4-2YV9', 'FDW6-KV8T', 'G6NX-V38F', 'G8NW-DFV2', 'R9RG-FXQB', 'S7L9-BK92', 'TK7R-TDM4', 'V88L-RJAL', 'VDX5-L9V4', 'VQ2A-WDQJ', 'WKA3-D4KR', 'WRA2-UVET', 'X9FD-ETRV', 'XR9M-7Y77', 'XVN4-JT2T', 'Y4ZC-XPYL', 'Y5KF-6V92', 'Y6A4-VELC', 'YTB3-5APS', 'ZDK5-TKFC', 'ZL8R-UZ5N', 'ZSH5-DMES'
rebeccacremona commented 3 years ago

All 44 links marked "deletion_required" had, in fact, never been uploaded to IA in the first place. Neither had the object marked "deletion_incomplete". If that happens again, there might be a bug in my status-setting logic, though I don't see one.