retorquere / zotero-better-bibtex

Make Zotero effective for us LaTeX holdouts
https://retorque.re/zotero-better-bibtex/
MIT License
5.36k stars 288 forks source link

biblatex export of ‘Presentation’: Use ‘type’ and ‘venue’ fields #644

Closed njbart closed 6 years ago

njbart commented 7 years ago

Prompted by https://github.com/retorquere/zotero-better-bibtex/issues/643, I had another look at the biblatex export of ‘Presentation’, and would like to suggest using ‘type’ and ‘venue’ fields when BBT exports such items to biblatex @unpublished entries.

To begin, I stand by my earlier recommendation to map Zotero’s ‘Meeting Name’ to biblatex eventtitle (rather than note).

Next, I found that BBT -> biblatex exports Zotero’s ‘Type’ both to type (capitalised if originally in lower case) and to howpublished (uncapitalised). I would recommend to export it to type only, and not capitalising it (since biblatex styles might want to render this somewhere in mid-sentence).

BBT should also map a Zotero ‘Presentation’’s ‘Place’ field to biblatex venue (rather than location), both because location in biblatex is always understood to be a publisher’s location, and because eventtitle might be empty, in which case the only way for pandoc-citeproc to identify a speech is by a non-empty venue field.

This biblatex format I’m recommending is well suited as an input format for pandoc-citeproc, survives the roundtrip Zotero/CSL -> biblatex -> CSL, does not mess with ‘Extra’/note, and also preserves the meaningful field names type and eventtitle rather than irretrievably discarding this information by using the too generic howpublished and note.

But (of course) there’s a bit of a snag:

Some popular biblatex styles (notably, biblatex-chicago and biblatex-apa) do recommend using the @unpublished entry type for speeches, but do not render type and eventtitle fields for this entry type. What they expect are complex expressions like note = {keynote address, annual meeting of the Council of Science Editors} (example from biblatex-chicago) or NOTE = {Paper presented at the Seventh International Conferene on Enterprise Information Systems} (example from biblatex-apa).

BBT, when using howpublished and note was able to approximate the former, but not the latter. (But note that the Chicago Manual itself contains a number of examples like ‘Paper presented at the Blah blah’, which cannot be rendered correctly using the ‘old’ approach either.)

Hence, I strongly prefer the ‘new’ approach (type and eventtitle) – even if this means that until biblatex styles are fixed (something I will propose to the style authors), or unless you want to offer a preference in BBT to choose between these two options (not something I would go for), biblatex users will have to map type and eventtitle to howpublished and note themselves, e.g. by including, in the latex document preamble,

\DeclareSourcemap{
  \maps[datatype=bibtex]{
    \map[overwrite]{
      \pertype{unpublished}
        \step[fieldsource=type, fieldtarget=howpublished]
        \step[fieldsource=eventtitle, fieldtarget=note]
    }
  }
}

Using biblatex’s \DeclareSourcemap, it is also possible to combine fields and add strings, e.g., note={Paper presented at the Foo} from type={paper} and eventtitle={Foo}. But I feel all this is best left for the users to decide.

javidcf commented 7 years ago

I think mapping "Place" to venue instead of location also makes sense for conference papers. Even if you may want both venue and location, I think venue is more relevant in this case.

Also a bit unrelated, but for conference papers the "Date" field is also exported as publication date instead of eventdate, although that is more complicated, because you will want the publication date (the year at least), which generally matches the event date but I guess not in 100% of cases... and also it would require handling date ranges.

retorquere commented 7 years ago

Date range support is pretty good in BBT.

I will only be able to address these things after the Z5 port. Addressing it will be hugely helped by samples exported in Z5 BBT JSON format and corresponding Bib(La)TeX samples covering the various cases (hint, hint).

javidcf commented 7 years ago

@retorquere I'd be happy to help with that, now or when the time for this comes. I'm already using the Z5 BBT test version (tbh works pretty great so far). I'm putting venue and eventdate with something like bibtex[eventdate=2000-01-01/2000-01-02;venue=City, Country] in "Extra" right now and it works. To be fair, even if it is not enormously convenient the fact is that putting the conference dates will always require some manual work, since I haven't seen any translator extract them (which makes sense, given that Zotero's "Date" field doesn't seem to support ranges...).

I can provide examples but I'm just not sure what these should be (that is, what should be the "right" mapping).

retorquere commented 7 years ago

I have no idea what mapping would be right; perhaps ironically, I'm a very unsophisticated bib(la)text user. I take the samples I get, try to make it work, add them to the test set; I rely on outside expertise to make sure the mapping actually makes sense. Sadly, my usual go-to guy for these things has dropped off the radar.

Zotero's date field doesn't support ranges, but BBT does its best to parse ranges out of that field; so 1980--1990 will be recognized as a date range. BBT5 will also support EDTF, which includes a formal syntax for ranges, which will make date parsing safer.

njbart commented 7 years ago

I think mapping "Place" to venue instead of location also makes sense for conference papers.

I don’t think that’s such a good idea. Zotero’s “conference papers” are understood to be formally published, and in Zotero, the fields “Place” and “Publisher” are mapped to the CSL variables “publisher-place” and “publisher” – with “publisher-place” corresponding most closely to biblatex’s “location”.

For “conference papers”, BBT should continue to map “Place” to biblatex’s “location”. A venue can be indicated by inserting the string “event-place: Some Venue” in Zotero’s “Extra” field (using citeproc-js’s “cheater syntax”); BBT should map this to biblatex’s “venue”.

retorquere commented 7 years ago

@njbart ! So happy to see you!

retorquere commented 7 years ago

Can someone get me a test case for the cheater-syntax venue by right-clicking a reference and sending a debug report?

retorquere commented 6 years ago

Anyone?

javidcf commented 6 years ago

I submitted an example with eventdate and venue in the Extra field some days ago I think.

retorquere commented 6 years ago

I can't see who submits what; I need the submission ID.

javidcf commented 6 years ago

Okay I submitted again and I saw it now, FA4YRARQ. I'm not sure I submitted it correctly the first time because I don't remember getting any ID...

retorquere commented 6 years ago

That would export to

@inproceedings{kingma2015,
  abstract = {We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments. The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. The method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. Empirical results demonstrate that Adam works well in practice and compares favorably to other stochastic optimization methods. Finally, we discuss AdaMax, a variant of Adam based on the infinity norm.},
  annotation = {Comment: Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015},
  author = {Kingma, Diederik P. and Ba, Jimmy},
  booktitle = {Proceedings of the 3rd {{International Conference}} on {{Learning Representations}} ({{ICLR}} 2015)},
  date = {2015},
  eventdate = {2015-05-07/2015-05-09},
  keywords = {Computer Science - Learning},
  title = {Adam: {{A Method}} for {{Stochastic Optimization}}},
  venue = {San Diego, CA, USA}
}
blip-bloop commented 6 years ago

:robot: this is your friendly neighborhood build bot announcing test build 4818 ("how did this pas before?").

retorquere commented 6 years ago

Can you try 4848?

blip-bloop commented 6 years ago

:robot: this is your friendly neighborhood build bot announcing test build 4900 ("New Crowdin translations (#806)\n\n New translations zotero-better-bibtex.dtd (French)\r\n\r\n New translations zotero-better-bibtex.dtd (French)").

retorquere commented 6 years ago

Please try 4900.

retorquere commented 6 years ago

Still interested?

javidcf commented 6 years ago

Yes, sorry, I'm not always at my regular work place... It seems to be working great, thanks.

retorquere commented 6 years ago

Thanks for the confirmation, will be part of .46

njbart commented 6 years ago

Not sure what happened, but it seems neither of the two suggestions from my OP have been dealt with after all: BBT -> biblatex still exports Zotero’s ‘Type’ both to type (capitalised if originally in lower case) and to howpublished (uncapitalised), as well as ‘Place’ to ‘location’.

retorquere commented 6 years ago

Hey @njbart, I'm trying to re-parse the OP, but it would really be easiest for me if you could assemble a few sample references and export them both as BBT JSON and (corrected as you see fit) BibLaTeX.

njbart commented 6 years ago

Here’s a relevant example from CMS 16e, with the expected biblatex output (bibtex export seems ok):

Exported Items.json Exported Items.bib

retorquere commented 6 years ago

Zotero also hase a "Meeting Name" field. Is that not more likely to be the venue?

retorquere commented 6 years ago

Is there a specific reason to exclude the rights field for presentations? It's present in the Zotero reference you gave me.

njbart commented 6 years ago

Zotero also has a "Meeting Name" field. Is that not more likely to be the venue?

No. According to the biblatex manual, “the location list holds the place of publication” (so it should be empty for “Presentation” and other unpublished types), and “The location of the event is given in the venue field.” And the current mapping Zotero "Meeting Name" -> biblatex “eventtitle” seems an excellent match, too.

Is there a specific reason to exclude the rights field for presentations? It's present in the Zotero reference you gave me.

Neither biblatex nor CSL have a rights field or anything similar (biblatex-chicago or biblatex-apa don’t have any such fields either), and Zotero itself never exports the GUI field “Rights”. So, though it certainly doesn’t do any harm, I see no need at all for BBT to export it.

retorquere commented 6 years ago

What do you make of this? If @murmlgrmpf is right, the type field would either need to be encoded as literal or it would have to have case conversion (as with the title field) applied. The biblatex manual says it's a key.

njbart commented 6 years ago

Unfortunately @murmlgrmpf did not give any details on the style used – but “PhD” to “Phd“ does not look like a standard title-case-to-sentence-case conversion, so apart from a hunch that it might be a weirdly configured style, I don’t really know what to think about this particular case.

The biblatex manual has type as a “key” field; so, unless the content matches a localization key, “the value is printed as is”. (And indeed, when I tested this with biblatex-apa, one of the more popular styles that converts titles to sentence case, PhD Thesis was not changed case-wise.)

So following the manual, and the behaviour of biblatex-apa, I’d say no case conversion and protection is needed here. (I’ll admit though that if use cases could be found that cause genuine problems with mainstream biblatex styles, I’d be willing to rethink it …)

retorquere commented 6 years ago

@murmlgrmpf I will need your feedback on this, otherwise I'm going to have to revert #435.

blip-bloop commented 6 years ago

:robot: this is your friendly neighborhood build bot announcing test build 5417 ("fixes #644").

retorquere commented 6 years ago

@njbart can you try 5417? This contains the reversal of #435 -- whether or not that gets checked in depends on whether @murmlgrmpf joins in.

njbart commented 6 years ago

5417 looks good so far.

retorquere commented 6 years ago

I'm giving it until tomorrow; if no further feedback rolls in, I'm merging this and cutting a release.

murmlgrmpf commented 6 years ago

sorry for the late response, I needed to reread the thread and the one I had opened. Unfortunately I do not exactly remember my latex settings but we had a minimal setup that reproduced the issue.

retorquere commented 6 years ago

Hey @murmlgrmpf, thanks for stopping by. I still have that minimal setup and it's part of my tests -- the question is not so much how to reproduce it, but why you wanted it. @njbart is my go-to guy for biblatex knowledge -- I know next to nothing about biblatex myself, ironically -- and he seems to say that the pre-#435 behavior was actually the correct behavior, and that the type field should never get case manipulation/brace protection. We're trying to find out what style you used that generated biblatex with that caused the unmodified PhD Thesis to render something you did not expect.

njbart commented 6 years ago

Seems like we’ve been talking at cross purposes – @murmlgrmpf most likely has been exporting to bibtex (though the #435 title says “biblatex” …), since BBT/biblatex normally exports a thesis to thesis while BBT/bibtex uses phdthesis, as in @murmlgrmpf’s example.

Now bibtex does indeed change the case of the type field (see example below), but biblatex does not.

\documentclass{article}
\usepackage[utf8]{inputenc}
\begin{filecontents}{\jobname.bib}
@phdthesis{phdthesis,
  author = {Doe, J.},
  title = {Title of a phdthesis item},
  type = {Type field: ALLCAPS Upper lower camelCase},
  year = {2015},
}
@techreport{techreport,
  author = {Doe, J.},
  title = {Title of a techreport item},
  type = {Type field: ALLCAPS Upper lower camelCase},
  year = {2015},
}
\end{filecontents}
\bibliographystyle{plain}                         % Try either of
%\usepackage{natbib} \bibliographystyle{plainnat} % these two lines.
\begin{document}
\section*{Text}
\cite{phdthesis, techreport}
\bibliography{\jobname}
\end{document}

with plain.bst:

screen shot

murmlgrmpf commented 6 years ago

@njbart you've got it right. My issue was, that the automatic build system of some Springer journal did not support biblatex. Therefore we rely on bibtex.

njbart commented 6 years ago

No offence, but a decent MWE would have saved us a lot of effort here …

retorquere commented 6 years ago

OK, so that makes it easy to fix, as the bibtex and biblatex exporters from BBT already exhibit different behavior. I've corrected the title of #435, as that did throw me off.

blip-bloop commented 6 years ago

:robot: this is your friendly neighborhood build bot announcing test build 5450 ("test for #435").

murmlgrmpf commented 6 years ago

There is a MWE in the original thread I had opened and it is still working according to @retorquere .... anyway thank's to all of you!

retorquere commented 6 years ago

The MWE might not have helped, because as noted, I really know incredibly little about bibtex/biblatex. What threw me off was that #435 explicitly said biblatex, so I changed the behavior for the biblatex exporter. It's now changed back, and the behavior is only in the bibtex exporter. The next release will have it -- I'm trying to get #891 and #892 closed and then I'll cut a new release.

murmlgrmpf commented 6 years ago

Thank you! Keep up the good work!

Emiliano Heyns notifications@github.com schrieb am Sa., 27. Jan. 2018, 13:09:

The MWE might not have helped, because as noted, I really know incredibly little about bibtex/biblatex. What threw me off was that #435 https://github.com/retorquere/zotero-better-bibtex/issues/435 explicitly said biblatex, so I changed the behavior for the biblatex exporter. It's now changed back, and the behavior is only in the bibtex exporter. The next release will have it -- I'm trying to get #891 https://github.com/retorquere/zotero-better-bibtex/issues/891 and #892 https://github.com/retorquere/zotero-better-bibtex/issues/892 closed and then I'll cut a new release.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/retorquere/zotero-better-bibtex/issues/644#issuecomment-360980636, or mute the thread https://github.com/notifications/unsubscribe-auth/ABZq27TrD9L8E9trmjv-WxI8uWerQx5mks5tOxHrgaJpZM4L-dPQ .

njbart commented 6 years ago

Well, I beg to differ: A (LaTeX) MWE would have helped a lot because it would have shown clearly whether it’s bibtex or biblatex that was being used.

retorquere commented 6 years ago

To you, yes :laughing:. I can hardly tell the difference, and I would almost certainly have missed from the MWE that it was bibtex and not biblatex. It's hard to overstate how little I know about bib(la)tex, or about LaTeX for that matter. I use it for pretty much all my writing, but I also heavily re-use documents because they once worked for me. If a document compilation breaks, I'm usually lost.

njbart commented 6 years ago

Ok, gentle intro: The telltale sign is the string biblatex appearing as (part of) a package name in the preamble of a LaTeX document – e.g.

\usepackage[style=authoryear]{biblatex}, \usepackage[authordate]{biblatex-chicago}, or \usepackage[style=apa]{biblatex}.

In the absence of that string, it must be bibtex (don’t look for the string bibtex though; all that bibtex requires in the preamble is the command \bibliographystyle{}, e.g., \bibliographystyle{plain}).

retorquere commented 6 years ago

That's manageable even for me :)

lock[bot] commented 6 years ago

This thread has been automatically locked because it has not had recent activity. Please open a new issue for related bugs and link to relevant comments in this thread.