`nocite`s not rendered

[njbart]

Citekeys placed in a nocite pandoc metadata field are not being rendered if using pandoc-zotxt.lua.

Example, using cite (with Zotero running, and containing a @brush:ornithology item):

pandoc --lua-filter pandoc-zotxt.lua -C -t plain << EOT
@brush:ornithology
EOT

actual = expected output:

Brush and Clark (1983)

Brush, A. H., and G. A. Clark Jr., eds. 1983. Perspectives in
Ornithology. Cambridge: Cambridge Univ. Press.

using nocite:

pandoc --lua-filter pandoc-zotxt.lua -C -t plain << EOT
---
nocite: '@brush:ornithology'
...
EOT

Actual output: nothing

Expected:

Brush, A. H., and G. A. Clark Jr., eds. 1983. Perspectives in
Ornithology. Cambridge: Cambridge Univ. Press.

For comparison, biblio data in pandoc metadata:

pandoc -C -t plain << EOT
---
nocite: '@brush:ornithology'
references:
- id: brush:ornithology
  editor:
    - family: Brush
      given: A. H.
    - family: Clark
      given: G. A.
      suffix: Jr.
  issued: 1983
  language: en-GB
  publisher: Cambridge Univ. Press
  publisher-place: Cambridge
  title: Perspectives in ornithology
  type: book
...
EOT

actual = expected output:

Brush, A. H., and G. A. Clark Jr., eds. 1983. Perspectives in
Ornithology. Cambridge: Cambridge Univ. Press.

I'd be happy to investigate further, e.g., how exactly nocites are represented in the pandoc AST, but maybe that isn't necessary, and there's an immediately obvious solution. Just let me know.

[odkr]

This is already fixed in the development branch. The functions that Pandoc provides for walking the AST simply don't cover metadata. I'm a bit busy right now, so it ~might~ will take a while before I'll come around to releasing a new beta.

[odkr]

But thanks a lot for being such an avid and accurate reporter! I appreciate your help 😃.

[odkr]

Pandoc v2.15 has made major change to Pandoc's Lua API. elem_type and walk no longer work as a consequence. Re-opening this for the time being.

[odkr]

I’ve released a beta version that searches for citations in all metadata fields. I’d welcome feedback and testing, you can download it at https://github.com/odkr/pandoc-zotxt.lua/releases/tag/v1.1.0b3.

[njbart]

Many thanks, but I’m afraid that after installing 1.1.0b3, my examples from the OP behave exactly as they did before, including no output at all for the nocite example.

(pandoc 2.16.2, Zotero 5.0.97-beta.57+07df7d0de, zotxt 5.1.4, BBT 5.6.11)

[odkr]

That’s odd. I'll need to have a look at my test case agin.

[njbart]

Just to double-check, I just tried this with the absolute path to the filter, but the result is unchanged, no output at all.

pandoc --lua-filter  /Users/nb/.local/share/pandoc/filters/pandoc-zotxt.lua-1.1.0b3/pandoc-zotxt.lua  -C -t plain << EOT
---
nocite: '@brush:ornithology'
...
EOT

[odkr]

I can't say that I have found the bug. But give the new beta a try. I can say that I cannot reproduce the issue with the beta and the most recent version of Zotero, so here's to hoping.

[njbart]

Finally, I seem to have figured out why I was seeing what I was seeing – I was using the dev version of pandoc (which identifies as "2.16.2", just like the released version). Two dev versions I tried, one compiled a few weeks ago and one compiled yesterday, behave identically in rendering no output at all in case of the nocite example.

When I switched to the latest release (2.16.2), binary installed via homebrew on macOS, pandoc-zotxt.lua started behaving as expected WRT the nocite example.

Of course it’d be interesting to know if the unexpected results with the latest dev can be reproduced by others.

Lastly, a hunch – I did spot quite a bit of activity having to do with lua components on https://github.com/jgm/pandoc/ lately, so it might be worth checking out whether anything significant pandoc-zotxt.lua is relying upon has changed.

[odkr]

Oh, that's interesting! And it makes a lot of sense. Pandoc moved to a new major release of HsLua, as a consequence the implementation details of Pandoc's own data types have changed -- and the elem_walk function I use to walk the metadata depends on those details, unfortunately. I am suprised that even more changes are incoming though. Thanks for the heads up!

[odkr]

Okay, metadata elements don't report any type at all in the development version of Pandoc. I doubt that this is intentional. I'll file a bug with them.

[odkr]

So, it is intentional. I adapted the script, but am not sure how well it works (the test suite doesn't work for non-released versions because it relies on the reported version number). If you want to give it a go, have a look at the release-1.1.0b5 branch, it contains the fix.

[njbart]

1.1.0b5 does seem to work as expected so far. In case I come across anything unusual, I'll let you know.

[odkr]

Thanks!

[odkr]

The pandoc.utils.type API has now been released and v1.1.0 uses it.