Postscript for improved BibDesk compatibility

[daaaaaaaaaniel]

Issue #667 discusses interoperability between BBT for Zotero and the open source macOS application BibDesk. The changes implemented in response to that discussion were aimed at allowing Zotero to import file attachments from a BibDesk-formatted .bib file.

I would like to do the reverse (as I described on the Zotero discussion forum). I'm trying to achieve this by using BBT's postscript feature to convert the attachment file paths into key-value pairs comprised of key bdsk-file-1 (and bdsk-file-2 etc.) and base64 encoded file path values.

This is the code I am using. It has not been successful and I am having a hard time debugging it. I tried to write it so that for every item with an attachment, it should first check if there already exists an entry for bdsk-file-1. If not, it should generate a base64 value based on the file path.

if (Translator.BetterBibTeX) {
  if (item.file) {
    if (!entry.has.bdsk-file-1) {
      var app = Application.currentApplication();
      app.includeStandardAdditions = true;
      var encodedPath = app.doShellScript('echo -n entry.file | openssl base64');
      // Zotero.debug(encodedPath);
      entry.add({name: 'bdsk-file-1', value: encodedPath});
    }
  }
}

Support log is AJM4RPPX-refs-euc.

[github-actions[bot]]

It looks like you did not upload an support log. The support log is important; it gives @retorquere your current BBT settings and a copy of the items under consideration as a test case so he can best replicate your issue, or build towards the desired behavior. Without it, @retorquere is effectively blind. Support logs are useful for both analysis and for enhancement requests; in the case of export enhancements, I need the copy of the references you have in mind.

If you did try to submit a support log, but the ID looked like D<number>, that is a Zotero debug report, which I cannot access. Please re-submit a BBT debug log by one of the methods below.

This request is much more likely than not to apply to you too @daaaaaaaaaniel, even if you think it unlikely. Please trust @retorquere when he says he will usually need one; he will more often than not just end up saying "please send a debug log". Let's just skip over the unnecesary delay this entails. Sending a debug log is very easy:

1. If your issue relates to how BBT behaves around a specific reference(s), such as citekey generation or export, select at least one of the items(s) under consideration, right-click it, and submit an BBT support log from that popup menu. If the problem is with export, please do include a sample of what you see exported, and what you expected to see exported for these references.

2. If the issue does not relate to references and is of a more general nature, generate an support log by restarting Zotero with debugging enabled (Help -> Debug Output Logging -> Restart with logging enabled), reproducing your problem, and selecting "Send Better BibTeX debug report..." from the help menu.

Once done, you will see a debug ID in red. Please post that debug id in the issue here.

Thank you!

[retorquere]

This may work for you:

if (Translator.BetterTeX && item.attachments) {
  item.attachments.forEach((att, i) => {
    entry.add({ name: `bdsk-file-${i + 1}`, value: btoa((att.saveFile ? att.defaultPath : '') || att.localPath) })
  })
}

[daaaaaaaaaniel]

Thanks @retorquere ! This doesn't quite work, but it definitely points in the right direction.

@tothzit in the Zotero forums has pointed out that the bdsk-file-n field does not need to be encoded as a base64 alias -- it works with regular file paths too. So I tried removing the btoa from the snippet provided above to make it output human-readable file paths, and discovered that a bunch of \ escaped characters had been inserted into the values of the bdsk-file-n field. The \ breaks the paths in filenames that include ie. "&" or "_". I don't know how to prevent this.

[retorquere]

if (Translator.BetterTeX && item.attachments) {
  item.attachments.forEach((att, i) => {
    entry.add({ name: `bdsk-file-${i + 1}`, value: (att.saveFile ? att.defaultPath : '') || att.localPath, enc: 'verbatim' })
  })
}

but note that verbatim escapes nothing, and unbalanced {..} braces will render the entry unparsable. I would bet that this and similar problems is why they chose base64 encoding. In what sense does that "not quite" work? You'd have to ask the bibdesk devs what they expect to find in such fields. It may be that they expect a specific encoding of the string-to-be-base64'ed.

[daaaaaaaaaniel]

Thanks, this enc: 'verbatim' trick works!

@retorquere I'm not the best person to answer your question re: what exactly is going on with BibDesk's alias files. But the explanation in this old thread is concise and provides more clarity. Ivan writes that it's a binary plist file encoded in base64, and decoding it gives a xml representation of what BibDesk stores.

This thread is helpful too. Glenn gives us a one-liner to decode the bdsk-file-n path.

echo "DATA" | base64 -D | plutil -convert xml1 -o - -

where "DATA" should be replaced with the actual contents of the field.

In my own repo you can see some sample encoded file paths output by BibDesk compared to their regular counterparts. Check out MyBibliography-ZoteroBBT-BibLaTeX.bib for the BBT output, and MyBibliography-ZoteroBBT-BibDesk.bib for the result after opening it in BibDesk and saving it (I trimmed off some references to make the file smaller).

To highlight just one example:

The regular filepath is /Users/daniel/Documents/Text Repository/ZotFile/Journal Article/Social Media + Society/vol. 7/no. 1/Abidin 2021 - From Networked Publics to Refracted Publics [abidin2021_Netw].pdf
After saving it in BibDesk, it becomes YnBsaXN0MDDSAQIDBFxyZWxhdGl2ZVBhdGhZYWxpYXNEYXRhXxCrLi4vLi4vRG9jdW1lbnRzL1RleHQgUmVwb3NpdG9yeS9ab3RGaWxlL0pvdXJuYWwgQXJ0aWNsZS9Tb2NpYWwgTWVkaWEgKyBTb2NpZXR5L3ZvbC4gNy9uby4gMS9BYmlkaW4gMjAyMSAtIEZyb20gTmV0d29ya2VkIFB1YmxpY3MgdG8gUmVmcmFjdGVkIFB1YmxpY3MgW2FiaWRpbjIwMjFfTmV0d10ucGRmTxEC4AAAAAAC4AACAAAMTWFjaW50b3NoIEhEAAAAAAAAAAAAAAAAAAAAAAAAAEJEAAH/////H0FiaWRpbiAyMDIxIC0gRnJvbSNGRkZGRkZGRi5wZGYAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAP////8AAAAAAAAAAAAAAAAAAgAIAAAKIGN1AAAAAAAAAAAAAAAAAAVuby4gMQAAAgC0LzpVc2VyczpkYW5pZWw6RG9jdW1lbnRzOlRleHQgUmVwb3NpdG9yeTpab3RGaWxlOkpvdXJuYWwgQXJ0aWNsZTpTb2NpYWwgTWVkaWEgKyBTb2NpZXR5OnZvbC4gNzpuby4gMTpBYmlkaW4gMjAyMSAtIEZyb20gTmV0d29ya2VkIFB1YmxpY3MgdG8gUmVmcmFjdGVkIFB1YmxpY3MgW2FiaWRpbjIwMjFfTmV0d10ucGRmAA4AoABPAEEAYgBpAGQAaQBuACAAMgAwADIAMQAgAC0AIABGAHIAbwBtACAATgBlAHQAdwBvAHIAawBlAGQAIABQAHUAYgBsAGkAYwBzACAAdABvACAAUgBlAGYAcgBhAGMAdABlAGQAIABQAHUAYgBsAGkAYwBzACAAWwBhAGIAaQBkAGkAbgAyADAAMgAxAF8ATgBlAHQAdwBdAC4AcABkAGYADwAaAAwATQBhAGMAaQBuAHQAbwBzAGgAIABIAEQAEgCyVXNlcnMvZGFuaWVsL0RvY3VtZW50cy9UZXh0IFJlcG9zaXRvcnkvWm90RmlsZS9Kb3VybmFsIEFydGljbGUvU29jaWFsIE1lZGlhICsgU29jaWV0eS92b2wuIDcvbm8uIDEvQWJpZGluIDIwMjEgLSBGcm9tIE5ldHdvcmtlZCBQdWJsaWNzIHRvIFJlZnJhY3RlZCBQdWJsaWNzIFthYmlkaW4yMDIxX05ldHddLnBkZgATAAEvAAAVAAIADf//AAAACAANABoAJADSAAAAAAAAAgEAAAAAAAAABQAAAAAAAAAAAAAAAAAAA7Y=
which we can decode as

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>aliasData</key>
    <data>
    AAAAAALgAAIAAAxNYWNpbnRvc2ggSEQAAAAAAAAAAAAAAAAAAAAAAAAAQkQAAf////8f
    QWJpZGluIDIwMjEgLSBGcm9tI0ZGRkZGRkZGLnBkZgAAAAAAAAAAAAAAAAAAAAAAAAAA
    AAAAAAAAAAAAAAAA/////wAAAAAAAAAAAAAAAAACAAgAAAogY3UAAAAAAAAAAAAAAAAA
    BW5vLiAxAAACALQvOlVzZXJzOmRhbmllbDpEb2N1bWVudHM6VGV4dCBSZXBvc2l0b3J5
    OlpvdEZpbGU6Sm91cm5hbCBBcnRpY2xlOlNvY2lhbCBNZWRpYSArIFNvY2lldHk6dm9s
    LiA3Om5vLiAxOkFiaWRpbiAyMDIxIC0gRnJvbSBOZXR3b3JrZWQgUHVibGljcyB0byBS
    ZWZyYWN0ZWQgUHVibGljcyBbYWJpZGluMjAyMV9OZXR3XS5wZGYADgCgAE8AQQBiAGkA
    ZABpAG4AIAAyADAAMgAxACAALQAgAEYAcgBvAG0AIABOAGUAdAB3AG8AcgBrAGUAZAAg
    AFAAdQBiAGwAaQBjAHMAIAB0AG8AIABSAGUAZgByAGEAYwB0AGUAZAAgAFAAdQBiAGwA
    aQBjAHMAIABbAGEAYgBpAGQAaQBuADIAMAAyADEAXwBOAGUAdAB3AF0ALgBwAGQAZgAP
    ABoADABNAGEAYwBpAG4AdABvAHMAaAAgAEgARAASALJVc2Vycy9kYW5pZWwvRG9jdW1l
    bnRzL1RleHQgUmVwb3NpdG9yeS9ab3RGaWxlL0pvdXJuYWwgQXJ0aWNsZS9Tb2NpYWwg
    TWVkaWEgKyBTb2NpZXR5L3ZvbC4gNy9uby4gMS9BYmlkaW4gMjAyMSAtIEZyb20gTmV0
    d29ya2VkIFB1YmxpY3MgdG8gUmVmcmFjdGVkIFB1YmxpY3MgW2FiaWRpbjIwMjFfTmV0
    d10ucGRmABMAAS8AABUAAgAN//8AAA==
    </data>
    <key>relativePath</key>
    <string>../../Documents/Text Repository/ZotFile/Journal Article/Social Media + Society/vol. 7/no. 1/Abidin 2021 - From Networked Publics to Refracted Publics [abidin2021_Netw].pdf</string>
</dict>
</plist>

[retorquere]

Blegh, that seems like a very poor choice. Still, good to know. I might add that to the importer.