Helm-bibtex does not work when the .bib file is modified

[jmburgos]

Hi Titus,

Recently I noticed the following.

• I open emacs, load an org-mode buffer and do helm-bibtex.
• After ~15 seconds the bibliography appears (is this time normal? I have close to 3 thousand references).
• If I do helm-bibtex again, the bibliography appears in no time... so far so good.
• Now, if I open by bibtex file and made changes (for example by adding a new bibtex entry), then when I do helm-bibtex again, nothing happens. I only get a blank screen, and the message "pattern:" in the minibuffer. If I type something, I get the message "Loading bibliography..." in the minibuffer for a while, then again "pattern:", and the main buffer remains blank. I am running emacs 24.4.1 on Centos07, and helm bibtex 20160320.1654. Any ideas?

Julian

[tmalsburg]

Hi Julian! First, 15 seconds are definitely not expected for a bibliography with just 3K entries. My bibliography has a bit more than 1K entries and initial loading takes perhaps 1.5 seconds. Subsequently, it takes less than 0.5 due to caching within helm-bibtex. Are you using the file field to reference PDF files? That may be a bit slower than the default scheme (BibTeX key + .pdf) but it should still not take 15 seconds.

Second, if the BibTeX file is changed, it has to be loaded anew and this should take as much time as the initial loading. When no candidates show up, this is usually because there is a syntax error in the BibTeX file. Are you sure, your BibTeX is correct?

Helm, unfortunately swallows errors that occur during loading, but you can get the error message by calling helm-bibtex-candidates directly. This should either return a list of entries, or produce an error message in the message buffer. Could you please try that and report back?

[jmburgos]

Thanks Titus. I may have an error somewhere, although running bibtex-validate gives me a "Buffer is syntactically correct" message, and the org-ref-build-full-bibliography generates a pdf with all my citations. When running helm-bibtex-candidates, I do not get an error message on the message buffer. I get just a "Loading bibliography... " message, and the following appears on the main buffer:

(helm-bibtex-candidates) (("⌘ 2010 A protocol for data exploration to avoid common statistical problems Zuur, A.F. and Ieno, E.N. and Elphick, C.S. article Zuur2010" ("=has-pdf=" . "⌘") ("year" . "2010") ("title" . "{A protocol for data exploration to avoid common statistical problems}") ("author" . "{Zuur, A.F. and Ieno, E.N. and Elphick, C.S.}") ("=type=" . "article") ("=key=" . "Zuur2010")) ("⌘ 2009 A Beginner's Guide to R Zuur, A.F. and Ieno, E.N. and Meesters, E.H.W.G. book Zuur2009a" ("=has-pdf=" . "⌘") ("year" . "2009") ("title" . "{A Beginner's Guide to R}") ("author" . "{Zuur, A.F. and Ieno, E.N. and Meesters, E.H.W.G.}") ("=type=" . "book") ("=key=" . "Zuur2009a")) ("⌘ 2009 Zuur, Alain F and Ieno, Elena N. and Walker, Neil J. and Saveliev, Anatoly A. and Smith, Graham M. Mixed Effects Models and Extensions in Ecology with R book Zuur2009" ("=has-pdf=" . "⌘") ("year" . "{2009}") ("author" . "{Zuur, Alain F and Ieno, Elena N. and Walker, Neil J. and Saveliev, Anatoly A. and Smith, Graham M.}") ("title" . "{Mixed Effects Models and Extensions in Ecology with R}") ("=type=" . "book") ("=key=" . "Zuur2009")) ("⌘ 2007 Zuur, A.F. and Ieno, E.N. and Smith, G.M. and Ebooks Corporation Analysing ecological data book Zuur2007" ("=has-pdf=" . "⌘") ("year" . "{2007}") ("author" . "{Zuur, A.F. and Ieno, E.N. and Smith, G.M. and Ebooks Corporation}") ("title" . "{Analysing ecological data}") ("=type=" . "book") ("=key=" . "Zuur2007")) ("⌘ 2004 Zuur, A.F. and Pierce, G.J. Common trends in northeast Atlantic squid time series article Zuur2004" ("=has-pdf=" . "⌘") ("year" . "{2004}") ("author" . "{Zuur, A.F. and Pierce, G.J.}") ("title" . "{Common trends in northeast Atlantic squid time series}") ("=type=" . "article") ("=key=" . "Zuur2004")) ("⌘ 2003 Zuur, Alain F and Fryer, RJ and Jolliffe, IT and Dekker, R and Beukema, JJ Estimating common trends in multivariate time series using dynamic factor analysis article Zuur2003a" ("=has-pdf=" . "⌘") ("year" . "{2003}") ("author" . "{Zuur, Alain F and Fryer, RJ and Jolliffe, IT and Dekker, R and Beukema, JJ}") ("title" . "{Estimating common trends in multivariate time series using dynamic factor analysis}") ("=type=" . "article") ("=key=" . "Zuur2003a")) ("⌘ 2003 Zuur, A.F. and Tuck, I.D. and Bailey, N. Dynamic factor analysis to estimate common trends in fisheries time series article Zuur2003" ("=has-pdf=" . "⌘") ("year" . "{2003}") ("author" . "{Zuur, A.F. and Tuck, I.D. and Bailey, N.}") ("title" . "{Dynamic factor analysis to estimate common trends in fisheries time series}") ("=type=" . "article") ("=key=" . "Zuur2003")) ("⌘ 2009 Zimmermann, N.E. and Yoccoz, N.G. and Edwards Jr, T.C. and Meier, E.S. and Thuiller, W. and Guisan, A. and Schmatz, D.R. and Pearman, P.B. Climatic extremes improve predictions of spatial patterns of tree species article Zimmermann2009" ("=has-pdf=" . "⌘") ("year" . "{2009}") ("author" . "{Zimmermann, N.E. and Yoccoz, N.G. and Edwards Jr, T.C. and Meier, E.S. and Thuiller, W. and Guisan, A. and Schmatz, D.R. and Pearman, P.B.}") ("title" . "{Climatic extremes improve predictions of spatial patterns of tree species}") ("=type=" . "article") ("=key=" . "Zimmermann2009")) ("✎ ⌘ 2009 Zieger, S and Stieglitz, T and Kininmonth, S Mapping reef features from multibeam sonar data using multiscale morphometric analysis article Zieger2009" ("=has-note=" . "✎") ("=has-pdf=" . "⌘") ("year" . "{2009}") ("author" . "{Zieger, S and Stieglitz, T and Kininmonth, S}") ("title" . "{Mapping reef features from multibeam sonar data using multiscale morphometric analysis}") ("=type=" . "article") ("=key=" . "Zieger2009")) ("⌘ 1999 Zhou, M. An objective interpolation method for spatiotemporal distribution of marine plankton article Zhou1999" ("=has-pdf=" . "⌘") ("year" . "{1999}") ("author" . "{Zhou, M.}") ("title" . "{An objective interpolation method for spatiotemporal distribution of marine plankton}") ("=type=" . "article") ("=key=" . "Zhou1999")) ("⌘ 2002 Zheng, X and Pierce, GJ and Reid, DG and Jolliffe, IT Does the North Atlantic current affect spatial distribution of whiting? Testing environmental hypotheses using statistical and GIS techniques article Zheng2002" ("=has-pdf=" . "⌘") ("year" . "{2002}") ("author" . "{Zheng, X and Pierce, GJ and Reid, DG and Jolliffe, IT}") ("title" . "{Does the North Atlantic current affect spatial distribution of whiting? Testing environmental hypotheses using statistical and GIS techniques}") ("=type=" . "article") ("=key=" . "Zheng2002")) ("⌘ 2012 Zhao, Mingwei and Li, Fayuan and Tang, Guo'an and others Optimal scale selection for DEM based slope segmentation in the Loess Plateau article Zhao2012" ("=has-pdf=" . "⌘") ("year" . "{2012}") ("author" . "{Zhao, Mingwei and Li, Fayuan and Tang, Guo'an and others}") ("title" . "{Optimal scale selection for DEM based slope segmentation in the Loess Plateau}") ("=type=" . "article") ("=key=" . "Zhao2012")) ...)

[jmburgos]

Strange... it seems that is working fine now. Still takes about 15 seconds to load. I have many entries with the "file" field (legacy from when I was using JabRef to manage my bib file).

[tmalsburg]

Hm, glad that it's working now. But too bad that we couldn't track down what the issue was.

Just out of curiosity: could you set helm-bibtex-pdf-field to nil and check how long it takes to load the library then? Make sure that you invalidate the cache to force a reread (see here).

[jmburgos]

It still takes 12-13 seconds, but it is not a big deal Now, if I edit the bib file it does not get stuck, which was the problem. I think that earlier probably I had a bad entry. In the last couple of days I went through the entries in added in the last weeks and cleaned up a few potentially problematic ones. That made the problem go away. Now I have no excuses to finish that paper.... :)

[jmburgos]

I should add that I have 2152 entries in my bibtex file... not 3 thousands as I though.

[tmalsburg]

Would you mind sharing your bibtex file with me? I'd be interested to see how long loading takes on my machine. 12-13 seconds for just 2152 sounds very strange to me. If it's also slow on my machine, I'd like to track down whether the problem lies in helm-bibtex or the library that we use to parse BibTeX. My email address is malsburg@ucsd.edu. Thank you!