erc-dharma / project-documentation

DHARMA Project Documentation
Creative Commons Attribution 4.0 International
3 stars 3 forks source link

attention to duplicate ZSTs #311

Open arlogriffiths opened 5 months ago

arlogriffiths commented 5 months ago

Yesterday I encountered this message:

Capture d’écran 2024-05-22 à 13 25 56

I did what I normally do, i.e. search the Zotero Short Title in Zotero standalone, merge the two items, and then synchronise, but it did not make the error go away. After consultation with @michaelnmmeyer, we have learned that merging two (or more) entries into one is not enough to make duplicates go away. The merge process relegates one (or more) of the previously existing versions to the Trash bin. It is necessary at a final step to 'permanently delete' the rejected version(s) from the Trash bin. Only then will our system cease to believe there is a problem of multiple items bearing the same short title.

@michaelnmmeyer — maybe we should document this matter in the next version of ZG?

This is a reminder to everyone to regularly take a look in the Duplicate Items bin and try to contribute to merging duplicates. (Attention, the Duplicate Items bin is sometimes misleading, as in some cases it suggests that items are duplicates whereas in reality they are not.) As final step of the process, I now realize, it is important also to remove rejected items from the Trash bin. See the bins at the bottom of the following screenshot.

Capture d’écran 2024-05-23 à 09 16 56
michaelnmmeyer commented 5 months ago

I have added an extra check to filter out entries that bear a 'deleted' flag when retrieving entries by short title. The same issue should thus not surface again (but this is not tested, I do not have enough info for reproducing it).

However, I still do not filter out entries that bear a 'deleted' flag when displaying the global bibliography (at https://dharmalekha.info/bibliography), because this is expensive to evaluate. I could find a workaround, but this would likely be useless, because ultimately I plan to only show cited entries in the global bibliography.

arlogriffiths commented 5 months ago

Thanks Michaël, but I am not sure that filter is necessary: it's desirable for us to be pushed to clean up Zotero as much as possible. Also, I fear the changes you have made turn out to have unwanted side-effects. Some perfectly good ZSTs are now not being recognized by our database:

Capture d’écran 2024-05-23 à 17 02 47

Please try to debug, or simply undo entirely the changes you made when implementing that filter.

michaelnmmeyer commented 5 months ago

Fixed in https://github.com/michaelnmmeyer/dharma/commit/6b3cdd4cf24f3d8cad9adefe9d92acd81fa5c7a8.

salomepichon commented 5 months ago

Dear all, Noted for the Zotero "duplicate items" and "trash bin" process.

Best,