Export web page to "misc" type with "notes" and "howpublished" custom fields

[jipodine]

Hi,

I am using a modified version of the Zotero BibTeX translator, which is inferior to yours, but simpler, and therefore easier to adapt to my needs.

I would like to switch to Better BibTeX, but I must be overlooking something, because I could not find anywhere in the options where to handle the web pages. Due to the poor support of BibTeX (and style files) for this, I generally use the "misc" category, with the last accessed date in the "notes" field, and the url in the "howpublished" field. Is there a way to handle it with Better BibTeX? (I know that Zotero dropped this, and this is precisely with I am using a modified translator...)

Best.

[retorquere]

Do you have a sample of what you want to achieve?

[gracile-fr]

That's only for Better BibTeX, right? I mean Better BibLaTeX use the @online type… (@retorquere: two labels "BibTex" / "BibLaTeX" might help)

FWIW: http://tex.stackexchange.com/questions/3587/how-can-i-use-bibtex-to-cite-a-web-page

[retorquere]

This can only be for BibTeX, right. I'm on the fence on how to best do this; on the one hand it can be done using bibtex[...], as publishing web pages this way is hackish in any case. On the other hand, if that reference at stack exchange you link to is the "best" way to do it, I might as well bite the bullet and just always export webpages that way.

[jipodine]

Yes, indeed, it is only for BibTeX, which is still required by almost any publisher (a least in my domain)... And what I need is to add custom howpublished and note fields to the @misc entries. These fields need to be custom, since they may change depending on the publisher. (I then regenerate a corresponding BibTeX file.)

The StackExchange example shows what I would like to achieve, but I do want to keep the other fields (which are sometime used, and already produced by the Zotero translator). See also the Wikipedia example, BibTeX entry

I my personal case, I also use the urldate field (in addition to the "note") for the last accessed date. I use the href package for the howpublished field (while others may prefer url or even nothing). And I use the journal field for the web-site, if relevant.

Here is an example of my current output (which uses author instead of journal, but all of this stays in Zotero):

@misc{ntp.org_clock_2015,
        title = {Clock {Quality}},
        url = {http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm},
        urldate = {2015-02-13},
        note = {(last accessed 2015-02-13)},
        author = {{NTP.org}},
        year = {2015},
        howpublished = {\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}},
        file = {NTP.org - 2015 - Clock Quality.html:/Users/jipodine/Documents/zotero/storage/BXUMNKVE/NTP.org - 2015 - Clock Quality.html:text/html}}
}

(As a side note, since UTF is not supported by BibTeX, I always escape everything. I noticed that the file field is not, but I think it could be ignored anyway.)

[jipodine]

And yes, it is hackish, but the "best" way because the only one that works with the old styles that are still required.

[gracile-fr]

If it's custom, have a look there: https://zotplus.github.io/better-bibtex/export.html#add-your-own-biblatex-fields but that's at the item-level and won't solve your problem if you've to change the fields depending on the publisher. If you actually need a way to customize the export (almost) each time, a custom translator is the best solution. The other solution would be to allow users of BBT to customize the export, but that's a lot of work.

[retorquere]

Customizing the export is not technically hard, but

1. I can't give any support other than helping to debug for translators that deviate from my own.
2. It requires rebuilding the extension since the translators are written in coffeescript, compiled to javascript, and the result is bundled inside the extension
3. Mozilla is going to make this impossible soon since extension will have to be signed, and changes in files inside the extension invalidate the signature, and the modified extension will not load.

Modifying the assembled BBT translator is technically possible (if you're going this way, have a mental health professional on speed-dial, and pick one BBT installs into the translators dir rather than from the BBT sources), and you could just drop it into your translators folder (don't forget to change the name and ID, or bad stuff will happen), but I'm afraid I cannot offer support for it beyond hints on how to get certain things done. If it breaks, it breaks.

I don't want to add a "add your ownpost-processing javascript", because that's going to be impossible to debug.

Per-item is is indeed possible using the page @gracile-fr links to, even if it will require the upcoming 1.2.35 to enable LaTeX inside those constructs (currently, everything in those constructs is escaped).

I don't currently have an idea how I would cleanly add per-publisher variability. Not against it in any way, but I'd need something clean, that doesn't add a bazillion preferences to the already extravagant prefs pane of BBT. The only thing that springs to mind is XSLT, but I wouldn't say that XSLT is especially easy to create (or debug).

[retorquere]

I could just add the "howpublished" field, subject to the "fancyURLs" setting, I don't see much harm in that. For the "notes" field it's a different matter, as it requires inserting text not entered into Zotero, which would probably require translation.

[retorquere]

@jipodine : for the entry you posted in https://github.com/ZotPlus/zotero-better-bibtex/issues/329#issuecomment-135977082, could you right-click on that in Zotero, select "report zotplus error", and post the resulting debug ID here? I may have a solution (of sorts).

[retorquere]

@jipodine the file field is not escaped as it is not used in bibliographies, but is used by JabRef and others to open the associated file. You might be interested in https://github.com/ZotPlus/zotero-ascii-rename.

[retorquere]

@jipodine I will need your help in getting a resolution.

[jipodine]

@retorquere : here is the debug ID: 7XIZE5P3. Thank you for looking at this, and the clarification about the filefield.

Of course, the solution that is needed here is a global per-collection, as I would rather use a post-processing script (on the resulting BibTeX file) than manually update every item in Zotero.

[retorquere]

I'm looking for a way to achieve this that doesn't require you to manually edit each entry, which doesn't make too many unwarranted assumptions on the output, and which doesn't assume everyone speaks English. On it, but not a quicky.

[retorquere]

Not released yet, but you can try to see if this is what you're after: install http://tempsend.com/8AE794DF08, go into about:config, and set preference extensions.zotero.translators.better-bibtex.postscript to

  if (this.item.itemType === 'webpage' && Translator.BetterBibTeX) {
    if (this.has.urldate) {
      this.add({ name: 'note', value: "(accessed " + this.has.urldate.value + ")" });
    }
    if (this.has.url) {
      this.add({ name: 'howpublished', bibtex: "{\\href" + this.has.url.bibtex + this.has.url.bibtex + "}" });
    }
  }

I'm still on the fence on the form in which I should release this. The code in the preference is ran for every non-cached reference after BBT has done all its work, and allows modifying the reference in place before it's flushed out. The problem with this is that you have to know the internal API to make it work, this API has exactly zero documentation, and I honestly don't feel like documenting all of it. Plus, this is almost guaranteed to require adjustment at some point as I sometimes do change the internal API.

[retorquere]

I've updated https://zotplus.github.io/better-bibtex/export.html with some documentation

[retorquere]

@jipodine are you still interested? I'd love to get verification that this solves your problem.

[jipodine]

Yes, of course, I am still interested :) Thank you for your good solution, as it allows customisation, and will less break anyway that changing parts of a generated script (which is what I am doing now)...

Some points need to be cleared, though... When I copied the script you provided in extensions.zotero.translators.better-bibtex.postscript, I got the following result, with nested \href tags:

@misc{ntp.org_clock_2015,
  title = {Clock {Quality}},
  url = {\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}},
  timestamp = {2015-02-13 17:15:15},
  urldate = {2015-02-13},
  author = {{NTP.org}},
  year = {2015},
  file = {NTP.org - 2015 - Clock Quality.html:/Users/lambert/Documents/zotero/storage/BXUMNKVE/NTP.org - 2015 - Clock Quality.html:text/html},
  note = {(accessed 2015-02-13)},
  howpublished = {\href{\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}}{\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}}}
}

I then tried without the script:

@misc{ntp.org_clock_2015,
  title = {Clock {Quality}},
  url = {\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}},
  timestamp = {2015-02-13 17:15:15},
  urldate = {2015-02-13},
  author = {{NTP.org}},
  year = {2015},
  file = {NTP.org - 2015 - Clock Quality.html:/Users/lambert/Documents/zotero/storage/BXUMNKVE/NTP.org - 2015 - Clock Quality.html:text/html}
}

So I guessed that there were 2 issues:

• the url is already encoded with the \href tag (as it appears in the url BibTeX field)
• there is an error in the documentation, as the href tag must be used with a url and a description

With the modified script:

if (this.item.itemType === 'webpage' && Translator.BetterBibTeX) {
    if (this.has.urldate) {
      this.add({ name: 'note', value: "(accessed " + this.has.urldate.value + ")" });
    }
    if (this.has.url) {
      this.add({ name: 'howpublished', bibtex: this.has.url.bibtex });
    }
  }

I finally got this result

@misc{ntp.org_clock_2015,
  title = {Clock {Quality}},
  url = {\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}},
  timestamp = {2015-02-13 17:15:15},
  urldate = {2015-02-13},
  author = {{NTP.org}},
  year = {2015},
  file = {NTP.org - 2015 - Clock Quality.html:/Users/lambert/Documents/zotero/storage/BXUMNKVE/NTP.org - 2015 - Clock Quality.html:text/html},
  note = {(accessed 2015-02-13)},
  howpublished = {\href{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}{http://www.ntp.org/ntpfaq/NTP-s-sw-clocks-quality.htm}}
}

While this is correct, I would suggest not to encode the url, so as one could choose to use \url, \href or no tag. Also, the documentation should be corrected (and could use the \url form, which is simpler).

[retorquere]

This is all up to the scripter (that is: you). The urls were already wrapped in hrefs because you have the "fancyURLs" set to true, and that is what they do. If you want to fiddle around with the URL itself, you can find it as this.has.url.value rather than this.has.url.bibtex. You can do something like

this.add({name: 'howpublished', value: this.has.url.value, enc: 'url'})

and that will follow the "fancyURLs" setting.

[retorquere]

This has already been release BTW.

[jipodine]

Everything is clear, now. So everything works fine, and only the documentation could be updated to use either your last form, that works according to the fancyURLs setting:

if (this.item.itemType === 'webpage' && Translator.BetterBibTeX) {
    if (this.has.urldate) {
      this.add({ name: 'note', value: "(accessed " + this.has.urldate.value + ")" });
    }
    if (this.has.url) {
      this.add({name: 'howpublished', value: this.has.url.value, enc: 'url'});
    }
  }

or this one that works no matter the fancyURLs setting:

if (this.item.itemType === 'webpage' && Translator.BetterBibTeX) {
    if (this.has.urldate) {
      this.add({ name: 'note', value: "(accessed " + this.has.urldate.value + ")" });
    }
    if (this.has.url) {
      this.add({ name: 'howpublished', bibtex: "{\\url{" + this.has.url.value + "}}" });
    }
  }

Here I used the \url tag, as it is easier, but note that the \href tag was broken in the doc anyway, as it should be:

this.add({ name: 'howpublished', bibtex: "{\\href{" + this.has.url.value + "}{" + this.has.url.value + "}}" });

Thanks a lot.

[retorquere]

The last two aren't correct; this.has.url.value needs to be verbatim, not raw, as not all valid URL characters are safe for verbatim. You could do that with

this.add({ name: 'howpublished', bibtex: "{\\url{" + this.enc_verbatim({value: this.has.url.value}) + "}}" });

I've opened a wiki page for examples; I'd appreciate it if you could add your example there. I'll consolidate into the web site perhaps at a later point.

[jipodine]

I did, for the wiki page. Please do check that this is correct (and update the documentation page).