Error when name contains brackets

failable commented 4 years ago

When the bibtex value contains brackets (or escape sequences?), citeproc-el emits error:

Debugger entered--Lisp error: (wrong-type-argument listp #<hash-table equal 44535/64414 0x1feb87b0a7a1>)
  assoc-default("LATIN SMALL LETTER S CEDILLA" #<hash-table equal 44535/64414 0x1feb87b0a7a1>)
  citeproc-bt--to-ucs("c" "s")
  #f(compiled-function (x) #<bytecode 0x1feb8ae76c59>)("\\c{s}")
  replace-regexp-in-string("\\\\\\(?1:[\"'.=^`~]\\)[[:space:]]*\\(?2:[[:alpha:]]\\)\\|..." #f(compiled-function (x) #<bytecode 0x1feb8ae76c59>) "Agirre, Eneko and Alfonseca, Enrique and Hall, Kei...")
  citeproc-bt--decode("Agirre, Eneko and Alfonseca, Enrique and Hall, Kei...")
  citeproc-bt--to-csl("\"Agirre, Eneko and Alfonseca, Enrique and Hall, Ke...")
  citeproc-bt-entry-to-csl((("=type=" . "inproceedings") ("=key=" . "agirre-2009-study-similarity") ("title" . "\"A study on similarity and relatedness using distr...") ("author" . "\"Agirre, Eneko and Alfonseca, Enrique and Hall, Ke...") ("booktitle" . "\"Proceedings of Human Language Technologies: The 2...") ("pages" . "\"19-27\"") ("year" . "2009") ("organization" . "\"Association for Computational Linguistics\"")))
  #f(compiled-function (key beg end) #<bytecode 0x1feb8b9ed355>)("agirre-2009-study-similarity" 156024 #<marker (moves after insertion) at 156664 in  *temp*>)
  bibtex-map-entries(#f(compiled-function (key beg end) #<bytecode 0x1feb8b9ed355>))
  #f(compiled-function (itemids) #<bytecode 0x1feb82cf6dc1>)(("tang-2005-email" "wei-2019-eda" "sennrich-2015-improv" "xie-2019-uda" "cubuk-2019-randaugment" "papineni-2002-bleu" "luong-2014-addres-rare" "wang-2015-learn-natur" "huang-2015-bidirect-lstm-crf" "sun-2019-how-fine" "yang-2019-bert-ad-hoc-doc" "wang-2019-multi-passage-bert" "ganitkevitch-2013-ppdb" "hill-2014-simlex" "agirre-2009-study-similarity" "weston-2015-towar-ai" "rajpurkar-2016-squad" "nguyen-2016-ms-marco" "fader-2013-parap" "bowman-2015-large-annotated" "bach-2017-learn-struc" "ratner-2016-data-progr" "ratner-2017-snorkel"))
  citeproc-proc-put-items-by-id(#s(citeproc-proc :style #s(citeproc-style :info (... ... ... ... ... ... ... ... ... ... ... ... ...) :opts (... ... ... ... ...) :bib-opts (... ... ... ... ...) :bib-sort (lambda ... ...) :bib-sort-orders (t t t) :bib-layout (lambda ... ...) :cite-opts (... ... ... ...) :cite-note nil :cite-sort nil :cite-sort-orders nil :cite-layout (lambda ... ...) :cite-layout-attrs (... ... ...) :locale-opts (... ...) :macros (... ... ... ... ... ... ... ... ... ... ... ...) :terms (... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) :uses-ys-var nil :date-text (... ... ... ...) :date-numeric (... ... ... ...)) :getter #f(compiled-function (itemids) #<bytecode 0x1feb82cf6dc1>) :itemdata #<hash-table equal 0/65 0x1feb8ae12add> :citations #s(queue :head nil :tail nil) :names #<hash-table equal 0/65 0x1feb8ae128ad> :finalized t) ("tang-2005-email" "wei-2019-eda" "sennrich-2015-improv" "xie-2019-uda" "cubuk-2019-randaugment" "papineni-2002-bleu" "luong-2014-addres-rare" "wang-2015-learn-natur" "huang-2015-bidirect-lstm-c..." "sun-2019-how-fine" "yang-2019-bert-ad-hoc-doc" "wang-2019-multi-passage-be..." "ganitkevitch-2013-ppdb" "hill-2014-simlex" "agirre-2009-study-similari..." "weston-2015-towar-ai" "rajpurkar-2016-squad" "nguyen-2016-ms-marco" "fader-2013-parap" "bowman-2015-large-annotate..." "bach-2017-learn-struc" "ratner-2016-data-progr" "ratner-2017-snorkel"))
  citeproc-append-citations((#s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (... ...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (... ...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil) #s(citeproc-citation :cites (...) :note-index nil :capitalize-first nil :suppress-affixes nil :grouped nil)) #s(citeproc-proc :style #s(citeproc-style :info (... ... ... ... ... ... ... ... ... ... ... ... ...) :opts (... ... ... ... ...) :bib-opts (... ... ... ... ...) :bib-sort (lambda ... ...) :bib-sort-orders (t t t) :bib-layout (lambda ... ...) :cite-opts (... ... ... ...) :cite-note nil :cite-sort nil :cite-sort-orders nil :cite-layout (lambda ... ...) :cite-layout-attrs (... ... ...) :locale-opts (... ...) :macros (... ... ... ... ... ... ... ... ... ... ... ...) :terms (... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...) :uses-ys-var nil :date-text (... ... ... ...) :date-numeric (... ... ... ...)) :getter #f(compiled-function (itemids) #<bytecode 0x1feb82cf6dc1>) :itemdata #<hash-table equal 0/65 0x1feb8ae12add> :citations #s(queue :head nil :tail nil) :names #<hash-table equal 0/65 0x1feb8ae128ad> :finalized t))
  citeproc-org--append-and-render-citations(((:elt ... :elt-no 0 :cite-no 0 :fn-no nil :new-fn nil) (:elt ... :elt-no 1 :cite-no 1 :fn-no nil :new-fn nil) (:elt ... :elt-no 2 :cite-no 2 :fn-no nil :new-fn nil) (:elt ... :elt-no 3 :cite-no 3 :fn-no nil :new-fn nil) (:elt ... :elt-no 4 :cite-no 4 :fn-no nil :new-fn nil) (:elt ... :elt-no 5 :cite-no 5 :fn-no nil :new-fn nil) (:elt ... :elt-no 6 :cite-no 6 :fn-no nil :new-fn nil) (:elt ... :elt-no 7 :cite-no 7 :fn-no nil :new-fn nil) (:elt ... :elt-no 8 :cite-no 8 :fn-no nil :new-fn nil) (:elt ... :elt-no 9 :cite-no 9 :fn-no nil :new-fn nil) (:elt ... :elt-no 10 :cite-no 10 :fn-no nil :new-fn nil) (:elt ... :elt-no 11 :cite-no 11 :fn-no nil :new-fn nil) (:elt ... :elt-no 12 :cite-no 12 :fn-no nil :new-fn nil) (:elt ... :elt-no 13 :cite-no 13 :fn-no nil :new-fn nil) (:elt ... :elt-no 14 :cite-no 14 :fn-no nil :new-fn nil) (:elt ... :elt-no 15 :cite-no 15 :fn-no nil :new-fn nil) (:elt ... :elt-no 16 :cite-no 16 :fn-no nil :new-fn nil) (:elt ... :elt-no 17 :cite-no 17 :fn-no nil :new-fn nil) (:elt ... :elt-no 18 :cite-no 18 :fn-no nil :new-fn nil) (:elt ... :elt-no 19 :cite-no 19 :fn-no nil :new-fn nil) ...) #s(citeproc-proc :style #s(citeproc-style :info ... :opts ... :bib-opts ... :bib-sort ... :bib-sort-orders ... :bib-layout ... :cite-opts ... :cite-note nil :cite-sort nil :cite-sort-orders nil :cite-layout ... :cite-layout-attrs ... :locale-opts ... :macros ... :terms ... :uses-ys-var nil :date-text ... :date-numeric ...) :getter #f(compiled-function (itemids) #<bytecode 0x1feb82cf6dc1>) :itemdata #<hash-table equal 0/65 0x1feb8ae12add> :citations #s(queue :head nil :tail nil) :names #<hash-table equal 0/65 0x1feb8ae128ad> :finalized t) html link nil)
  citeproc-org-render-references(html)
  run-hook-with-args(citeproc-org-render-references html)
  org-export-as(html nil nil nil (:output-file "/Users/user/org/public_html/notes/202004081953..." :base-directory "/Users/user/org/notes" :base-extension "org" :publishing-directory "/Users/user/org/public_html/notes/" :publishing-function (org-html-publish-to-html) :recursive t :auto-sitemap t :sitemap-title "Notes" :sitemap-filename "index.org" :sitemap-file-entry-format "%d - %t" :sitemap-sort-files anti-chronologically :sitemap-style list :sitemap-function org-publish-org-sitemap :sitemap-format-entry org-publish-org-sitemap-format :html-head-include-default-style nil :html-head-include-scripts nil :html-link-use-abs-url nil :crossrefs nil :filter-final-output (org-publish--store-crossrefs org-publish-collect-index)))
  org-export-to-file(html "/Users/user/org/public_html/notes/202004081953..." nil nil nil nil (:base-directory "/Users/user/org/notes" :base-extension "org" :publishing-directory "/Users/user/org/public_html/notes/" :publishing-function (org-html-publish-to-html) :recursive t :auto-sitemap t :sitemap-title "Notes" :sitemap-filename "index.org" :sitemap-file-entry-format "%d - %t" :sitemap-sort-files anti-chronologically :sitemap-style list :sitemap-function org-publish-org-sitemap :sitemap-format-entry org-publish-org-sitemap-format :html-head-include-default-style nil :html-head-include-scripts nil :html-link-use-abs-url nil :crossrefs nil :filter-final-output (org-publish--store-crossrefs org-publish-collect-index)))
  org-publish-org-to(html "/Users/user/org/notes/20200408195303-natural_l..." ".html" (:base-directory "/Users/user/org/notes" :base-extension "org" :publishing-directory "/Users/user/org/public_html/notes/" :publishing-function (org-html-publish-to-html) :recursive t :auto-sitemap t :sitemap-title "Notes" :sitemap-filename "index.org" :sitemap-file-entry-format "%d - %t" :sitemap-sort-files anti-chronologically :sitemap-style list :sitemap-function org-publish-org-sitemap :sitemap-format-entry org-publish-org-sitemap-format :html-head-include-default-style nil :html-head-include-scripts nil :html-link-use-abs-url nil) "/Users/user/org/public_html/notes/")
  org-html-publish-to-html((:base-directory "/Users/user/org/notes" :base-extension "org" :publishing-directory "/Users/user/org/public_html/notes/" :publishing-function (org-html-publish-to-html) :recursive t :auto-sitemap t :sitemap-title "Notes" :sitemap-filename "index.org" :sitemap-file-entry-format "%d - %t" :sitemap-sort-files anti-chronologically :sitemap-style list :sitemap-function org-publish-org-sitemap :sitemap-format-entry org-publish-org-sitemap-format :html-head-include-default-style nil :html-head-include-scripts nil :html-link-use-abs-url nil) "/Users/user/org/notes/20200408195303-natural_l..." "/Users/user/org/public_html/notes/")
  org-publish-file("/Users/user/org/notes/20200408195303-natural_l..." ("notes-source" :base-directory "/Users/user/org/notes" :base-extension "org" :publishing-directory "/Users/user/org/public_html/notes/" :publishing-function (org-html-publish-to-html) :recursive t :auto-sitemap t :sitemap-title "Notes" :sitemap-filename "index.org" :sitemap-file-entry-format "%d - %t" :sitemap-sort-files anti-chronologically :sitemap-style list :sitemap-function org-publish-org-sitemap :sitemap-format-entry org-publish-org-sitemap-format :html-head-include-default-style nil :html-head-include-scripts nil :html-link-use-abs-url nil) t)
  org-publish-projects((("notes" :components ("notes-source" "notes-static"))))
  org-publish(("notes" :components ("notes-source" "notes-static")) nil)
  funcall-interactively(org-publish ("notes" :components ("notes-source" "notes-static")) nil)
  call-interactively(org-publish record nil)
  command-execute(org-publish record)
  counsel-M-x-action("org-publish")
  ivy-call()
  ivy-read("M-x " ("toggle-debug-on-error" "find-library" "text-mode" "customize-group" "describe-char" "hs-hide-all" "visual-line-mode" "find-function" "ivy-bibtex" "profiler-start" "tree-sitter-hl-mode" "straight-pull-package" "run-python" "org-cliplink" "org-ref-webib-arxiv-get-pdf-add-bibtex-entry-from-..." "dired-rsync" "counsel-colors-emacs" "shell" "bibtex-entry-rename-key-at-point" "customize-face" "set-buffer-file-coding-system" "gitignore" "load-theme" "man" "flush-lines" "org-pandoc-export-to-markdown_strict" "recentf-cleanup" "find-function-on-key" "org-mode" "org-store-link" "view-mode" "delete-char" "find-variable" "explain-pause-mode" "org-roam-find-file" "lsp-ui-peek-find-references" "org-version" "emacs-version" "add-dir-local-variable" "org-roam-db-build-cache" "lsp" "csv-mode" "describe-face" "json-mode" "org-roam-mode" "company-diag" "presentation-mode" "org-ref-webib-acl-get-pdf-add-bibtex-entry-from-fi..." "spell-fu-mode" "list-faces-display" ...) :predicate #f(compiled-function (x) #<bytecode 0x1feb86a14abd>) :require-match t :history counsel-M-x-history :action counsel-M-x-action :keymap (keymap (67108908 . counsel--info-lookup-symbol) (67108910 . counsel-find-symbol)) :initial-input nil :caller counsel-M-x)
  counsel-M-x()
  funcall-interactively(counsel-M-x)
  call-interactively(counsel-M-x nil nil)
  command-execute(counsel-M-x)

Related bibtex item:

@inproceedings{agirre-2009-study-similarity,
  title        = "A study on similarity and relatedness using distributional
                  and wordnet-based approaches",
  author       = "Agirre, Eneko and Alfonseca, Enrique and Hall, Keith and
                  Kravalova, Jana and Pa{\c{s}}ca, Marius and Soroa, Aitor",
  booktitle    = "Proceedings of Human Language Technologies: The 2009 Annual
                  Conference of the North American Chapter of the Association
                  for Computational Linguistics",
  pages        = "19-27",
  year         = 2009,
  organization = "Association for Computational Linguistics"
}

andras-simonyi commented 4 years ago

Hi, thanks for reporting this! I've committed a change (dbab443) which should fix the problem.

failable commented 4 years ago

Hello, thanks for the quick fix.

I have a big bibtex file here and some org files each citing some of the bib items. When I do org-publish to publish my notes to HTML, I keep getting error like the one mentioned above...

That seems quite a lot... 😅

Do I need to add all these to citeproc-bt--comm-letter-to-ucs-alist?

andras-simonyi commented 4 years ago

I think you are not using the patch, please update your citeproc-el to the latest version in MELPA. citeproc-bt--comm-letter-to-ucs-alist only contains the most common accent combinations for speed-up on older Emacs versions, other combinations should also be handled by the code. In fact I've tried out the new version with your big bibtex file (on Emacs 27) and all entries were rendered without any problem except one where the the key format was problematic. (Ended with a '+' sign, I'll need to investigate this, but no relation to the present issue.) Cool bibliography BTW :-)

failable commented 4 years ago

I evaluated the latest code after I fetched it from github when I tested. But after I restart emacs and test again the issue is gone. Since the original issue is solved I will close the issue. Thanks!

andras-simonyi / citeproc-el

Error when name contains brackets #12