Run citeproc after lua filters

[newhallroad]

Thanks for a wonderful project. I use it all the time.

I sometimes need to run citeproc after lua filters. For example, the filter pandoc-zotxt needs to run before citeproc because citeproc acts on a bibliography file generated by the filter. (The filter generates a references file from citation keys in the input file, and then adds the references file as "bibliography" in the pandoc metadata.) This works fine from the cli if I run, for example,

> pandoc test.md -o test.html --standalone --lua-filter=pandoc-zotxt.lua --citeproc

However, it does not work from emacs pandoc-mode. In the hydra menu, whether I add the filter before or after I specify "Process citations" to "yes", citeproc is listed first in the settings preview and runs before the filter.

I have tried adding "citeproc" to the lua filter list, but it is not recognized as a filter and throws an error.

This seems similar to #104 but I didn't find a solution there. I am using pandoc on windows, installed via the installer provided by pandoc. I am using pandoc-mode version 20230416.952 from melpa.

[newhallroad]

I found a workaround, I think, which is to write a lua filter that effectively runs the same as citeproc:

function Pandoc (doc)
  return pandoc.utils.citeproc(doc)
end

Because pandoc-mode respects the order of lua-filters, this can be run at any point, either after or before other filters as desired. It's a workaround because the citeproc option in pandoc-mode has to be set, counterintuitively, to "no", and there is the extra step of writing and storing the citeproc lua filter. But it works pretty well until a better solution is suggested.

If this is the best option for now, updating the manual might be helpful for others encountering the same problem.

Thank you for your fantastic work.

[joostkremers]

Hi, sorry for the late reply. It's been a while since I looked at the code for pandoc-mode, but if I understand #104 correctly, pandoc-mode should now put filters before all other options on the command line (except the input and output formats), so isn't it already doing what you need?

If you could post your pandoc-mode settings, it would help me reproduce the problem.

[newhallroad]

Thank you for responding. And thanks for a wonderful mode that I use all the time.

You are right that pandoc-mode runs regular filters first. But it seems that lua filters specifically (which appropriately have their own list in the hydra menu) are still run at the end. That causes problems for filters like pandoc-zotxt.lua that has to run before citeproc. To be fair, this is a fairly specific use case, but I wonder if there are others. Perhaps it would make sense to make sure that lua filters, like regular filters, are run before other options? (I'm not sure if that would have any detrimental effects or not.)

In the meantime, the workaround I suggested above (using the citeproc.lua filter rather than the citeproc argument) is effective.

[joostkremers]

Right, filters and Lua filters are not the same thing... I do think it would make sense to treat them alike and run them both before other options. I'm not aware of anything that would break, but of course that's no guarantee. 🤔

If something does break, I'll have to come up with a way to specify the order of filters vs. options, and perhaps also the order of filters among themselves... But for now, I'll just push a fix to treat Lua filters the same as other filters.

[newhallroad]

Thanks so much. That should do the trick. I look forward to trying it out.

[joostkremers]

Well, I finally got round to this... Apologies for the long wait. Commit 340da87 should provide this.

[newhallroad]

Thank you! Sorry for taking a while to get back to you. I just got around to updating the package and it works like a dream. I'm not sure of GitHub etiquette. Is it up to me to close the comment now?

[joostkremers]

I don't know if there's a GitHub etiquette for closing issues... :smile: I'll just close it with this comment.