Figure captions not working with Pandoc citations enabled

[hummuscience]

I was trying out ox-hugo (thanks for this amazing package btw!) and tried to export some citaitons as generated by org-ref. The links themselves seem to works pretty nice but the generated bibliography is not so nice. Below is an example of my input and output.

Maybe using https://github.com/andras-simonyi/citeproc-org could help? I tried it using html-export and it did a beautiful job!

Input:

Lets have a look at citations.  This cites the book: Kandel, Principles of Neural Sciences cite:kandel2013

This cites a journal articel: cite:terenzio2017

This cites two articles: cite:hensel1960,hensel1951

bibliographystyle:unsrt
bibliography:~/cloud/phd/papers/bibliography.bib

Output:

Lets have a look at citations. This cites the book: Kandel, Principles of Neural Sciences kandel2013

This cites a journal articel: terenzio2017

This cites two articles: hensel1960,hensel1951
Bibliography

[kandel2013] Kandel, Principles of Neural Science, McGraw-Hill (2013). ↩

[terenzio2017] Terenzio, Schiavo & Fainzilber, Compartmentalized Signaling in Neurons: From Cell Biology to Neuroscience, Neuron, 96(3), 667-679 (2017). doi. ↩

[hensel1960] Hensel & Boman, AFFERENT IMPULSES IN CUTANEOUS SENSORY NERVES IN HUMAN SUBJECTS, Journal of Neurophysiology, 23(5), 564-578 (1960). doi. ↩

[hensel1951] Hensel & Zotterman, ACTION POTENTIALS OF COLD FIBRES AND INTRACUTANEOUS TEMPERATURE GRADIENT, Journal of Neurophysiology, 14(5), 377-385 (1951). doi. ↩

I think, generally, it is doing an OK job, but it could be better formatted. citeproc-org does that part nicely, I wonder if it can be leveraged here

[kaushalmodi]

Glad that this package works for you except for this issue.

I never got to trying org-ref. It's on my list. There's a known issue with org-ref and ox-md (ox-hugo extends from ox-md) that I opened a while back: https://github.com/jkitchin/org-ref/issues/558.

Though, I did add Pandoc citation support recently (needs having pandoc binary in your PATH). That works great from my moderate testing: https://ox-hugo.scripter.co/doc/pandoc-citations/.

So while org-ref gets fixed, see if Pandoc citations are something that you'd like to try.

[hummuscience]

Apparently Pandoc should also understand org-ref citations according to some reports and also according to the pandoc manual so theoretically it should recognize them via citeproc.

Nevertheless, I tried the pandoc route by simply changing the cite: to @ and adapting the properties of the subheading as follows:

:PROPERTIES:
:EXPORT_FILE_NAME: writing-your-labbook-in-org
:EXPORT_DATE: 2018-08-19
:EXPORT_HUGO_MENU: :menu "main"
:EXPORT_HUGO_PANDOC_CITATIONS: t
:EXPORT_BIBLIOGRAPHY: ~/cloud/phd/papers/bibliography.bib
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :link-citations true
:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :foo bar :baz zoo :alpha 1 :beta "two words" :gamma 10
:END:

I get the following error when exporting:

pandoc: Unknown extension: fenced_divs

[kaushalmodi]

pandoc: Unknown extension: fenced_divs

I'd have to guess that your pandoc version is older than one where that extension got added.

Can you try updating to the latest Pandoc release?

https://github.com/jgm/pandoc/releases

The binaries are statically linked. So you don't need to "install" them. Just download, extract, and put the binaries in one of the directories in your PATH.

[hummuscience]

For some reason homebrew had 5 versions of pandoc installed and then there was another one installed manually so I had to use the uninstallation script provided by pandoc to get rid of it.

Then I needed to both install pandoc and pandoc-citeproc throught homebrew (I am on a Mac) to make it work.

While the citations seem to render (after an additional refresh), this rendering seems to mess with some other functionality. In my case, captions of code blocks/figures and the embedding of figures. Here is an example of what happens: https://imgur.com/a/XOseVlT

And here is the post that generates it:

** TODO Writing your Lab Notebook in Org                            :org:lab:
:PROPERTIES:
:EXPORT_FILE_NAME: writing-your-labbook-in-org
:EXPORT_DATE: 2018-08-19
:EXPORT_HUGO_MENU: :menu "main"
:EXPORT_HUGO_PANDOC_CITATIONS: t
:EXPORT_BIBLIOGRAPHY: /Users/muadabdelhay/cloud/phd/papers/bibliography.bib
:EXPORT_HUGO_CUSTOM_FRONT_MATTER+: :link-citations true
:EXPORT_HUGO_CUSTOM_FRONT_MATTER: :foo bar :baz zoo :alpha 1 :beta "two words" :gamma 10
:END:

*** Why an electronic lab notebook?
At an age where data is moving from paper to electornic form, the amount and *shape* of data is not very paper-compatible, and the analysis of that same data becomes more complex, it is just obvious that the move to an electronic notebook has to happen.
*** Why use org-mode?
Check out the code block in ref:test

Lets have a look at citations.  This cites the book: Kandel, Principles of Neural Sciences @kandel2013

This cites a journal articel: @terenzio2017

This cites two articles: @hensel1960,hensel1951

*** Using code blocks

#+name: clock-test
#+caption: test of a code block
#+begin_src R :results output graphics :exports results :file-ext png :res 180 :width 2400 :height 900 :noweb yes

plot(mtcars)
#+end_src

#+RESULTS: clock-test
#+caption: test for a caption of result file
[[file:clock-test.png]]

[kaushalmodi]

Update: As we later learn, below was not the actual issue.

---

Earlier:

About the captions, that's a different issue. Hugo does not render Markdown in figure shortcode captions for whatever reason. So we need to override that figure shortcode in the site/theme.

For example, put this version of the shortcode in your site/theme: https://github.com/kaushalmodi/hugo-onyx-theme/blob/master/layouts/shortcodes/figure.html .

I'm already tracking this issue here: https://github.com/kaushalmodi/ox-hugo/issues/136.

[kaushalmodi]

I'd appreciate if you can open an issue yourself in Hugo issues and link back here. I've mentioned this quite a few times there, but my opinion about this is going unheard.

[hummuscience]

The captions do work, if there is no pandoc conversion happening. The moment the citation are rendered, the captions break.

I will make an issue in Hugo for that

[kaushalmodi]

The moment the citation are rendered, the captions break.

Hmm, I need to try out your test case once I'm at my computer.

[kaushalmodi]

So don't open the issue on Hugo yet.

[kaushalmodi]

@Cumol Thanks for that example. I can reproduce the problem. Seems like it happens when Pandoc rewrites ox-hugo generated Markdown. I'll investigate.

[kaushalmodi]

While I investigate this bug.. a side note:

In this snippet:

#+name: clock-test
#+caption: test of a code block
#+begin_src R :results output graphics :exports results :file-ext png :res 180 :width 2400 :height 900 :noweb yes

plot(mtcars)
#+end_src

#+RESULTS: clock-test
#+caption: test for a caption of result file
[[file:clock-test.png]]

how did you add the second caption, the one in results: #+caption: test for a caption of result file? If you manually entered it, won't it get removed the next time you eval the code snippet?

If that caption got entered automatically, I wonder if that's specific to just ob-R (because I haven't coded in R and so never used ob-R).

[hummuscience]

I added it manually. Re-running the code would remove it.

I figured that if I write a blog post, at some point I won't be evaluating the code block anymore and I could add a caption to the linked image. So it is something one has to do just before finally publishing a post.

[kaushalmodi]

I just tried the below with HTML export, and it exports the same image twice. So manually adding an image to Org Babel created image is not expected as per Org design it seems:

#+name: code__plantuml_nested_boxes
#+caption: Nested Boxes
#+begin_src plantuml :file nested-boxes.svg :eval yes :exports both
rectangle "<html>, <body>, etc." as a  {
  rectangle "<div>..." as b #antiquewhite {
    rectangle "<video>...\n\n\n" as c
  }
}
#+end_src

#+RESULTS: code__plantuml_nested_boxes
#+caption: Figure caption
[[file:nested-boxes.svg]]

So for now, I will ignore the "manually addition of caption" part of your example.

[kaushalmodi]

The bug fix that I am focusing on is this:

The "before" (red part) is what ox-hugo generates. The "after" (green part) is after Pandoc rewrites that HTML span tag.

[kaushalmodi]

@Cumol Can you try this change: https://github.com/kaushalmodi/ox-hugo/commit/c035cd9dceb21b134d4d1e73ff16f49ec8ed5afb

It basically adds -bracketed_spans to org-hugo-pandoc-cite-pandoc-args-list.

[hummuscience]

It did not solve it for me. Still getting this:

[Code Snippet 1:]{.src-block-number} test of a code block

But the caption/image problem is solved.

[kaushalmodi]

I modified a defvar, so just re-evaluating the buffer won't work. You need to C-M-x the defvar form.. or with the updated ox-hugo-pandoc-cite.el, just restart Emacs.

[kaushalmodi]

Then I needed to both install pandoc and pandoc-citeproc throught homebrew (I am on a Mac) to make it work.

Thank you. I have added check for pandoc-citeproc too, and have updated docs at https://ox-hugo.scripter.co/doc/pandoc-citations/.

Also, thanks for your example. Based on that and this, I learned of a way to set caption for Org Babel generated figure, and also retain it after re-eval of the code block. I have updated my test with that. So now it closes resembles your original example. See https://ox-hugo.scripter.co/test/posts/citations-with-captions/.

But the caption/image problem is solved.

I am not sure how that could have got solved. But it's definitely solved after the fix in https://github.com/kaushalmodi/ox-hugo/commit/703823f827139add2f379872aa752b9a5b7a4684 :)

---

Thanks for opening this issue and bringing it to my attention. This helped fix few things:

• Pandoc now keeps the HTML span tags as-is.
• The escaping of < and > in wrapped shortcodes did not happen; it now happens.
• Such wrapped shortcodes are then "unwrapped" by the package; else Hugo would fail. So this means that figure shortcodes with captions will now work fine when using Pandoc.

[hummuscience]

I just updated ox-hugo through MELPA and now everything works as supposed. Thank you for fixing it :)

I wonder if we can get org-ref running through pandoc too as it seems to be supported (below a quote from the manual:

"The org-ref package is in wide use to handle citations and has excellent tooling support in Emacs. Its citation syntax is geared towards users in the natural sciences but still very flexible regardless."

[kaushalmodi]

I wonder if we can get org-ref running through pandoc too as it seems to be supported (below a quote from the manual:

I don't use org-ref. After that unresolved org-ref issue with ox-md, I didn't try further.

I'll though, give it a try again but with a lesser urgency.

This issue thread dealt with those bug fixes. So, can you please open a new thread with:

• an example Org file with the org-ref syntax you use
• expected Markdown file that that Org file should export to

?

From what I understand, org-ref should have just worked regardless of the exporter used. So I'm not sure what I should do different to make org-ref "work". But may be we get an answer to that once you post the example Org file.

I'll close this thread so that the org-ref discussion can start afresh in that new thread.