nondefault Data directory location

z5tron commented 6 years ago

I got over 1000 items tagged "#broken_attachments", for every one manually checked, I can double click the attachment to open it or right click to open its locations (Show file).

I guess it might be related to my setting: "Edit/Preferences/Advanced/Files and Folders/Data Directory Location", i.e. my "zotero"( of course the zotero/storage/XXXXXX) are D:\zotero.

retorquere commented 6 years ago

Are these linked files or stored files?

z5tron commented 6 years ago

both stored files and linked files, in my case, they are in different driver.

blip-bloop commented 6 years ago

:robot: this is your friendly neighborhood build bot announcing test build 5.0.7.29.issue-10 ("more debugging").

retorquere commented 6 years ago

The linked build will have more info, can you install that, and:

go into Help/Debug output logging/start with debugging enabled
after restart, scan again, go back into Help/Debug output logging, view log, and attach (not paste!) the content here

I don't really know what's going on, I ask Zotero for the attachment path; if it can't find the file, it gives me no path, and then I mark the attachment. If it can find the path, it unmarks the attachment. I'm just hoping there's some additional error in the log that explains some kind of side effect.

z5tron commented 6 years ago

I first deleted the "#broken_attachments" and "#duplicate_attachments" from lower left panel of Zotero, then do a new scan. Now the #broken_attachments tag appears again, when I select it, some items show up. I picked one book, go to the attachment, "Show File" brings up its real location in File Explorer. I feel confused this "#broken_attachments" is not really broken.

Then I go to the log file, find the following part, I am confused again: "StorageScanner.save: false" means the "#broken_attachments" is not saved at all.

(3)(+0000058): StorageScanner.attachment: {"itemID":8824,"path":"attachments:_/crilly_2008_50_mathematical_ideas_you_really_need_to_know.pdf","duplicates":1}

(3)(+0000058): Loading primaryData for 1 item in My Library

(4)(+0000056): SELECT O.itemID, O.itemTypeID, O.dateAdded, O.dateModified, O.libraryID, O.key, O.version, O.synced, COALESCE(CASE (SELECT COUNT(*) FROM itemCreators IC LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1) WHEN 0 THEN NULL WHEN 1 THEN (SELECT lastName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1) WHEN 2 THEN (SELECT (SELECT lastName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1) || ' and ' || (SELECT lastName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1,1)) ELSE (SELECT (SELECT lastName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1) || ' et al.' ) END, CASE (SELECT COUNT(*) FROM itemCreators WHERE itemID=O.itemID AND creatorTypeID IN (3)) WHEN 0 THEN NULL WHEN 1 THEN (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3)) WHEN 2 THEN (SELECT (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1) || ' and ' || (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1,1) ) ELSE (SELECT (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1) || ' et al.' ) END, CASE (SELECT COUNT(*) FROM itemCreators WHERE itemID=O.itemID AND creatorTypeID IN (2)) WHEN 0 THEN NULL WHEN 1 THEN (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2)) WHEN 2 THEN (SELECT (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1) || ' and ' || (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1,1) ) ELSE (SELECT (SELECT lastName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1) || ' et al.' ) END) AS firstCreator, COALESCE(CASE (SELECT COUNT(*) FROM itemCreators IC LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1) WHEN 0 THEN NULL WHEN 1 THEN (SELECT lastName || ' ' || firstName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1) WHEN 2 THEN (SELECT (SELECT lastName || ' ' || firstName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1,1)) ELSE (SELECT (SELECT lastName || ' ' || firstName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 1,1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators IC NATURAL JOIN creators LEFT JOIN itemTypeCreatorTypes ITCT ON (IC.creatorTypeID=ITCT.creatorTypeID AND ITCT.itemTypeID=O.itemTypeID) WHERE itemID=O.itemID AND primaryField=1 ORDER BY orderIndex LIMIT 2,1)) END, CASE (SELECT COUNT(*) FROM itemCreators WHERE itemID=O.itemID AND creatorTypeID IN (3)) WHEN 0 THEN NULL WHEN 1 THEN (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3)) WHEN 2 THEN (SELECT (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1,1) ) ELSE (SELECT (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 1,1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (3) ORDER BY orderIndex LIMIT 2,1)) END, CASE (SELECT COUNT(*) FROM itemCreators WHERE itemID=O.itemID AND creatorTypeID IN (2)) WHEN 0 THEN NULL WHEN 1 THEN (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2)) WHEN 2 THEN (SELECT (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1,1) ) ELSE (SELECT (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 1,1) || ' ' || (SELECT lastName || ' ' || firstName FROM itemCreators NATURAL JOIN creators WHERE itemID=O.itemID AND creatorTypeID IN (2) ORDER BY orderIndex LIMIT 2,1)) END) AS sortCreator, DI.itemID IS NOT NULL AS deleted, PI.itemID IS NOT NULL AS inPublications, (CASE O.itemTypeID WHEN 14 THEN IAP.itemID WHEN 1 THEN INoP.itemID END) AS parentID, (CASE O.itemTypeID WHEN 14 THEN IAP.key WHEN 1 THEN INoP.key END) AS parentKey, CS.charset AS attachmentCharset, IA.linkMode AS attachmentLinkMode, IA.contentType AS attachmentContentType, IA.path AS attachmentPath, IA.syncState AS attachmentSyncState, IA.storageModTime AS attachmentSyncedModificationTime, IA.storageHash AS attachmentSyncedHash, FI.guid AS feedItemGUID, FI.readTime AS feedItemReadTime, FI.translatedTime AS feedItemTranslatedTime FROM items O LEFT JOIN itemAttachments IA USING (itemID) LEFT JOIN items IAP ON (IA.parentItemID=IAP.itemID) LEFT JOIN itemNotes INo ON (O.itemID=INo.itemID) LEFT JOIN items INoP ON (INo.parentItemID=INoP.itemID) LEFT JOIN deletedItems DI ON (O.itemID=DI.itemID) LEFT JOIN publicationsItems PI ON (O.itemID=PI.itemID) LEFT JOIN charsets CS ON (IA.charsetID=CS.charsetID) LEFT JOIN feedItems FI ON (FI.itemID=O.itemID) WHERE 1 AND O.libraryID=? AND O.itemID IN (8824) [1]

(3)(+0000180): Loaded primaryData in My Library in 236 ms

(3)(+0000056): Loading creators for 1 item in My Library

(4)(+0000062): SELECT itemID, creatorID, creatorTypeID, orderIndex FROM items LEFT JOIN itemCreators USING (itemID) WHERE libraryID=? AND itemID IN (8824) ORDER BY itemID, orderIndex [1]

(3)(+0000118): Loaded creators in My Library in 180 ms

(3)(+0000064): Loading itemData for 1 item in My Library

(4)(+0000060): SELECT itemID, fieldID, value FROM items JOIN itemData USING (itemID) JOIN itemDataValues USING (valueID) WHERE libraryID=? AND itemTypeID!=? AND itemID IN (8824) [1, 1]

(4)(+0000108): SELECT itemID FROM items WHERE libraryID=? AND itemID IN (8824) [1]

(4)(+0000120): SELECT itemID, title FROM items JOIN itemNotes USING (itemID) WHERE libraryID=? AND itemID NOT IN (SELECT itemID FROM itemAttachments) AND itemID IN (8824) [1]

(3)(+0000106): Loaded itemData in My Library in 394 ms

(3)(+0000064): Loading note for 1 item in My Library

(4)(+0000058): SELECT itemID, note FROM items JOIN itemNotes USING (itemID) WHERE libraryID=? AND itemID IN (8824) [1]

(4)(+0000152): SELECT itemID FROM items WHERE libraryID=? AND itemID IN (8824) AND itemTypeID IN (?, ?) AND itemID NOT IN (SELECT itemID FROM itemNotes) [1, 1, 14]

(3)(+0000108): Loaded note in My Library in 316 ms

(3)(+0000064): Loading childItems for 1 item in My Library

(4)(+0000058): SELECT parentItemID, A.itemID, value AS title, CASE WHEN DI.itemID IS NULL THEN 0 ELSE 1 END AS trashed FROM itemAttachments A JOIN items I ON (A.parentItemID=I.itemID) LEFT JOIN itemData ID ON (fieldID=110 AND A.itemID=ID.itemID) LEFT JOIN itemDataValues IDV USING (valueID) LEFT JOIN deletedItems DI USING (itemID) WHERE libraryID=? AND parentItemID IN (8824) ORDER BY parentItemID [1]

(4)(+0000100): SELECT parentItemID, N.itemID, title, CASE WHEN DI.itemID IS NULL THEN 0 ELSE 1 END AS trashed FROM itemNotes N JOIN items I ON (N.parentItemID=I.itemID) LEFT JOIN deletedItems DI USING (itemID) WHERE libraryID=? AND parentItemID IN (8824) ORDER BY parentItemID [1]

(4)(+0000108): SELECT itemID FROM items I WHERE libraryID=? AND itemID IN (8824) AND itemID NOT IN (SELECT itemID FROM itemAttachments UNION SELECT itemID FROM itemNotes) [1]

(3)(+0000116): Loaded childItems in My Library in 382 ms

(3)(+0000062): Loading tags for 1 item in My Library

(4)(+0000062): SELECT itemID, name, type FROM items LEFT JOIN itemTags USING (itemID) LEFT JOIN tags USING (tagID) WHERE libraryID=? AND itemID IN (8824) [1]

(3)(+0000102): Loaded tags in My Library in 164 ms

(3)(+0000066): Loading collections for 1 item in My Library

(4)(+0000064): SELECT itemID, collectionID FROM items LEFT JOIN collectionItems USING (itemID) WHERE libraryID=? AND itemID IN (8824) [1]

(3)(+0000154): Loaded collections in My Library in 218 ms

(3)(+0000058): Loading relations for 1 item in My Library

(4)(+0000058): SELECT itemID, predicate, object FROM items LEFT JOIN itemRelations USING (itemID) LEFT JOIN relationPredicates USING (predicateID) WHERE libraryID=? AND itemID IN (8824) [1]

(3)(+0000100): Loaded relations in My Library in 158 ms

(3)(+0000104): StorageScanner.addTag('#broken_attachments ', false); hasTag: false

(3)(+0000066): StorageScanner.addTag('#duplicate_attachments', false); hasTag: false

(3)(+0000056): StorageScanner.save: false

retorquere commented 6 years ago

Can you get me a new log with .35?

blip-bloop commented 6 years ago

:robot: this is your friendly neighborhood build bot announcing test build 5.0.7.35 ("Merge branch '11' into 10").

z5tron commented 6 years ago

I found I have updated to 5.0.8, and the "#broken_attachments" are all correct now. Thanks for the fix.

There are one problem with "duplicate_attachments" detection, but I will start a new issue soon.

retorquere / zotero-storage-scanner

nondefault Data directory location #10