Citr not parsing author surnames containing several words

profandyfield commented 6 years ago

I'm working on a document that I worked on a few weeks ago and am getting errors that I didn't get a few weeks back (i.e. before the update to R 3.5.1.) I am using citr with zotero and the better bibtex plugin. When citr loads my library direct from zotero it fails to parse any reference that has an author surname containing more than one word. For example, I get a string or errors like this in the console:

Ignoring entry ‘de_lijster_age_2017’ (line944) because: 
    The name list field author cannot be parsed

Ignoring entry ‘dagani_meta-analysis_2017’ (line962) because: 
    The name list field author cannot be parsed

The citations all have in common that the surname is made up of several words. For example, for these two the corresponding zotero export is:

@article{de_lijster_age_2017,
  title = {The Age of Onset of Anxiety Disorders},
  volume = {62},
  issn = {0706-7437},
  doi = {10.1177/0706743716640757},
  number = {4},
  journal = {Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie},
  author = {{de Lijster}, Jasmijn M. and Dierckx, Bram and Utens, Elisabeth M.W.J. and Verhulst, Frank C. and Zieldorff, Carola and Dieleman, Gwen C. and Legerstee, Jeroen S.},
  year = {2017},
  pages = {237-246},
  file = {/Volumes/Alpha Lacertae/Documents/Academic/Library/zotero/storage/LIDTTYVM/de Lijster et al. - 2017 - The Age of Onset of Anxiety Disorders.pdf},
  pmid = {27310233},
  pmcid = {PMC5407545}
}

@article{dagani_meta-analysis_2017,
  title = {Meta-Analysis of the {{Interval}} between the {{Onset}} and {{Management}} of {{Bipolar Disorder}}},
  volume = {62},
  issn = {0706-7437},
  doi = {10.1177/0706743716656607},
  number = {4},
  journal = {Canadian Journal of Psychiatry. Revue Canadienne de Psychiatrie},
  author = {Dagani, Jessica and Signorini, Giulia and Nielssen, Olav and Bani, Moira and Pastore, Adriana and {de Girolamo}, Giovanni and Large, Matthew},
  year = {2017},
  pages = {247-258},
  file = {/Volumes/Alpha Lacertae/Documents/Academic/Library/zotero/storage/NL8Y8BTG/Dagani et al. - 2017 - Meta-analysis of the Interval between the Onset an.pdf},
  pmid = {27462036},
  pmcid = {PMC5407546}
}

So the first contains {de Lijster} as surname and the second {de Girolamo}.

If I save the above zotero export as a .bib file (see test.txt), reference it in the YAML header, fire up citr and disconnect it from zotero then citr reads the file perfectly. This suggests to me that the issue is to do with the way citr communicates with zotero and gets it to export so that it can load the database. I have literally no idea how that works though so I don't know whether this is a zotero issue, a BetterBibtex issue or a Citr issue. Any ideas of how to fix it or who I should be directing this at?

thanks andy

> sessionInfo()
R version 3.5.1 (2018-07-02)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS High Sierra 10.13.6

Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib

locale:
[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] shiny_1.1.0

loaded via a namespace (and not attached):
 [1] Rcpp_0.12.17     rstudioapi_0.7   xml2_1.2.0       magrittr_1.5     MASS_7.3-50      gmodels_2.18.1   xtable_1.8-2    
 [8] R6_2.2.2         rlang_0.2.1      bibtex_0.4.2     stringr_1.3.1    httr_1.3.1       plyr_1.8.4       tools_3.5.1     
[15] miniUI_0.1.1.1   gtools_3.8.1     htmltools_0.3.6  yaml_2.1.19      assertthat_0.2.0 digest_0.6.15    RefManageR_1.2.0
[22] later_0.7.3      promises_1.0.1   curl_3.2         mime_0.5         gdata_2.18.0     stringi_1.2.3    compiler_3.5.1  
[29] citr_0.2.0       forcats_0.3.0    jsonlite_1.5     lubridate_1.7.4  httpuv_1.4.4.2

profandyfield commented 6 years ago

Further digging reveals that behaviour is different when the surname starts with a lowercase letter than when it begins with an uppercase letter. For example, De Houwer imports to citr from zotero but de Houwer does not. Using two references of this author that are stored in my library using these two variants of the surname I looked at what Zotero does at export:

@article{de_houwer_extrinsic_2003,
  title = {The Extrinsic Affective {{Simon}} Task},
  volume = {50},
  issn = {1618-3169},
  doi = {10.1027//1618-3169.50.2.77},
  journal = {Experimental Psychology},
  author = {De Houwer, J.},
  year = {2003},
  pages = {77-85}
}

@article{de_houwer_potential_2008,
  title = {Potential Pitfalls in the Study of Associative Learning of Implicit Attitudes},
  volume = {43},
  issn = {0020-7594},
  journal = {International Journal of Psychology},
  author = {{de Houwer}, J.},
  year = {Jun-Aug 2008},
  pages = {396-396}
}

When the de begins with an upper case letter it is exported as {De Houwer, J.} but when it starts with a lower case d it is exported as {{de Houwer}, J.}. So, I suspect the extra brackets are throwing things off (is this assumption reasonable). my next question is, how do I stop this behaviour?

a.

crsh commented 6 years ago

Thanks for digging into this! I had a look at the latter example you posted. citr uses RefManageR::ReadBib() to read the references exported by Better Bib(La)TeX. The error message you are seeing is from ReadBib().

I saved the two references as a bib-file and also added them to my Zotero library (via import from clipboard). Using the current development version of citr (devtools::install_github("crsh/citr")) I was able to read both the file and the entries added to Zotero. I also got no errors when I read the file with ReadBib() directly.

To get a better idea of what's going on some additional diagnosing might help:

Does the problem persist with the development version of citr?
What version of Better Bib(La)Tex are you using (I'm using 5.0.180)?
How did you export the above entries? Did you choose, for example, Better BibLaTex or BibLaTeX format? Do you get the same formatting of the entries when you request them via http://localhost:23119/better-bibtex/library?/1/library.bibtex (this may take some time to load)?
Do you get the error when you save the above references to a file and read them with ReadBib()?

crsh commented 6 years ago

Sorry, I posted the wrong localhost-URL. The correct one is http://localhost:23119/better-bibtex/library?/1/library.bibtex (I also updated my previous post).

profandyfield commented 6 years ago

Thanks for the reply.

I'll check with the dev version of citr in due course, I'm going to try updating my mactex install first.
Also using 5.0.180
Exported by selecting references, right click export and selecting Better BibTex from the dropdown:

2018-07-27_10-59-47

RefManageR::ReadBib("test_2.bib") correctly reads this file (i had to change from .bib to .txt to upload here, but ReadBib processes both file extensions):

[1] J. De Houwer. “The Extrinsic Affective Simon Task”. In: _Experimental Psychology_ 50 (2003), pp. 77-85. ISSN:
1618-3169. DOI: 10.1027//1618-3169.50.2.77.

[2] J. de Houwer. “Potential Pitfalls in the Study of Associative Learning of Implicit Attitudes”. In: _International
Journal of Psychology_ 43 (), pp. 396-396. ISSN: 0020-7594.

I'll add more once mactex has updated ...

andy

profandyfield commented 6 years ago

Updating to macTex 2018 didn't help
Using the dev version of citr everything is fine! Hooray!

thanks

crsh commented 6 years ago

Great! And thanks for the detailed reports.

crsh / citr

Citr not parsing author surnames containing several words #35