Duplicate records created when importing files as linked attachments from previously imported folder

[somelinguist]

Hi. I've installed and setup rmlint following the instructions in #28, but this doesn't work when the imported folder contains items previously imported as linked attachments (since the files aren't actually duplicates).

Is there a way the add-on can check if there are already Zotero records that reference files in the folder to be imported, and not import those files if such a record exists?

[retorquere]

That's going to be difficult; rmlint doesn't take a job file, so I would have to add all existing linked files separately as command line parameters, and then we'd run into command line length limitations really quickly, which would lead to hard to debug problems.

If rmlint (or a cross-platform alternative that can write its output to a parseable file) could take a job spec input file it'd be easy, but you'd have to convince the rmlint author.

[github-actions[bot]]

:robot: this is your friendly neighborhood build bot announcing test build 0.0.7.38 ("rmlint.linked")

Install in Zotero by downloading test build 0.0.7.38, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

this adds a new boolean rmlint.linked which will scan de directories where the linked files reside. If there aren't too many different directories this may well work. Let me know.

[somelinguist]

Thanks so much for working on this. I tried the new version and setrmlint.linked to true, but it looks like it's still re-importing all files in the directory again, creating duplicate records of ones that had already been imported.

For context, I'm trying it out in a subfolder of my linked attachments base directory. That base directory is in iCloud Drive. Would that make a difference?

Are there any output logs to look into for the add-on and/or rmlint?

Thanks again!

[github-actions[bot]]

:robot: this is your friendly neighborhood build bot announcing test build 0.0.7.39 ("add baseAttachmentPath")

Install in Zotero by downloading test build 0.0.7.39, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[somelinguist]

Just tried test build 0.0.7.39, and unfortunately , there seems to be no change.

I looked at the last commit, and I was wondering what the format of the results in the DB query look like?

I looked at my zotero.sqlite file, and my linked attachments are stored with a raw format like: attachments:subfolder/filename.pdf

Does the query return the results with the actual path filled in, or do they still have the attachments: placeholder?

Thanks again!

[github-actions[bot]]

:robot: this is your friendly neighborhood build bot announcing test build 0.0.7.40 ("log folders to scan")

Install in Zotero by downloading test build 0.0.7.40, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

It excludes all paths that start with attachment:, as those files are in the storage directory, and I'm passing that path directly to rmlint.

Can you install build 40, enable debug logging from the help menu, try again, and then "send log to file.io" from the help menu?

[somelinguist]

Hi, Sorry for not getting back sooner. I was sick for about week when you posted the last message, and I just saw this when going through my notifications. I've installed the build and did the test as you described. This is the ID it gave me when I sent the log: 7R2VG6b68Chr-MKEGZWD2

Thanks!

[github-actions[bot]]

:robot: this is your friendly neighborhood build bot announcing test build 0.0.7.41 ("logging")

Install in Zotero by downloading test build 0.0.7.41, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

[retorquere]

No worries, life happens. Can you try again with build 41? And please also update the other plugins from the addins menu, Better BibTeX is a few versions behind and it's very chatty in the logs.