Correctly support org-cite functionality

a-fent commented 2 years ago

The upcoming 9.5 version of Org will support a native citation syntax. Make sure this is supported.

Don't think it's worth trying to support older org-ref citations, as org-ref's citation functionality is fully superseded by oc. (see https://github.com/kawabata/ox-pandoc/issues/77)

a-fent commented 2 years ago

With 9.5 installed, this creates a bit of a problem, because the BIBLIOGRAPHY keyword is processed and set by org mode itself, and set to a list (where ox-pandoc expects a string). Need some way to signal how to process citations (fully processed by org-mode or passed through to pandoc).

a-fent commented 2 years ago

[X] Prevent error with incompatible #+BIBLIOGRAPHY option
[X] biblatex processor passes through cite correctly
[x] CSL processor produces formatted output in intermediate org
[ ] pandoc processor (new) passes through pandoc citation

pank commented 2 years ago

FWIW the branch with support for oc works well enough for me. I am using it for org->docx with oc-csl for citations.

a-fent commented 2 years ago

Thank you for the feedback. I have picked the change that stops problems with Org 9.5 and #+BIBLIOGRAPHY and pushed it to the main branch, together with updated information in the README.org.

I'll keep this thread open to track remaining and new issues with citations.

bdarcus commented 2 years ago

Just linking this:

https://github.com/jgm/pandoc/issues/7329

EmmanuelCharpentier commented 2 years ago

I stumbled on this issue while googling for ways of exporting Org documents to Word/LibreOffice (the "standard" for text in my corporate salt mine) while keeping the link to Zotero (the bibliographic tool I managed to get tolerated n said salt mine...). Thois seems a good solution.

Just linking this:

jgm/pandoc#7329

Thanks to this page, I have been able to figure out and test successfully the export of a Markdown document to LibreOffice (and almost successfully to Word : I still have to manually add the reference list in Word...).

But I couldn't figure out what to use in a Org document. I think I have managed a correct set of options and metadata, but I can't get Org to place a reference in LibreOffice.

Did someone wrote this up for simple mortal users ?

To illustrate : the following :

# Essai de biblio exportable dans ODT/Word depuis Org

#+title: Essai Org
#+author: Emmanuel
#+date: 5 août 2022
#+language: fr
#+pandoc_options: lua-filter:./zotero.lua
#+pandoc_metadata: zotero_client:zotero
#+pandoc_metadata: zotero_author-in-text:false
#+pandoc_metadata: zotero_csl_style:vancouver-brackets
#+pandoc_metadata: zotero_cite-method:csl

* Essai de biblio /depuis *ORG* !/

Baratin citation [cite: @charpentierFullyBioresorbableDrugeluting2015] . Autre baratin.

* Références

results in :

Essai3.odt

where there is some marker for the reference list but northing recognizable as a reference.

For reference : here is the Marrkdown test and its results (using the same reference...) :

Essai1.docx Essai1.md Essai1.odt

Any hint ?

a-fent commented 2 years ago

Could you explain or offer a link to what all the filter / zotero stuff is doing here? I'm not familiar with it.

For citing to non-latex formats (like ODT) I would suggest using CSL and Org's own citation support. I find it works just adding two header lines:

#+BIBLIOGRAPHY: path/to/bibliography.bib
#+CITE_EXPORT: csl /path/to/csl-style.csl

and then exporting with org-pandoc-export-to-odt it works OK. I use Better Biblatex to export my Zotero collections to .bib.

EmmanuelCharpentier commented 2 years ago

Dear Alex,

Thank you for your answer. Here are some explanations :

Your mode of citation is the "normal" way to cite in a markup-based system (LaTeX, Markdown, Org, etc...). But unusable to users limited to a "word processor", where the "normal" modus operandi is to use a plugin importing references from a (front end to a) bibliographic database, such as Zotero (more and more popular...).

Better Bibtex allows to export citations

as .bib(tex) files (for "normal" citation in a markup language), but also
as pointers to a Zotero database (see here),

and Markdown is supposed to be able to make use of them.

My Test1 tested this and (mostly) validated this claim. What I want to do is to do the same thing from Org.

Using this on Org is (sort of) documented in a pandoc issue (long thread...), in which Bruce D'arcus claimed to have ported some documents from Markdown to Org with perfect results...

Using this type of citation would allow Word/LibreOffice users of an Org-originated document to work with the bibliography as if generated by the Zotero plugin. Big gain for us working in corporate salt mines wher "text" means "Word" and arkup languages are considered obscene.

Hence my interest in using this from Org...

bdarcus commented 2 years ago

Using this on Org is (sort of) documented in a https://github.com/jgm/pandoc/issues/7329(long thread...), in which Bruce D'arcus https://github.com/jgm/pandoc/issues/7329#issuecomment-998916589 to have ported some documents from Markdown to Org with "perfect results".

To clarify, that had nothing to do with your request; I just meant pandoc converted markdown citations to org-cite citations correctly.

What you're asking for is different: you want the pandoc export here to be able to be configured so that the ODT output uses Zotero-linked fields, instead of default rendered text.

FWIW, I agree this is useful feature; am just not sure where best it should be implemented (Pandoc? Org itself? here? all of these?).

a-fent commented 2 years ago

@EmmanuelCharpentier Thanks - I'm familiar with the problem you describe - I write papers in Org but co-authors use a mix of markdown, Word, ODT and Google docs. I have wanted a way to round-trip documents and possibly convert tracked edits into patches or diffs but haven't found a good solution.

I hadn't seen the BB information about creating live Zotero references via pandoc - very interesting. I imagine this would work with calling pandoc directly but not with ox-pandoc because ox-pandoc re-exports to org format first and then calls pandoc on this intermediate file. [cite:...] is already resolved in this first phase.

Please feel free to open a separate issue for this and point to your comments here. This issue is really more to track support for core org-cite in ox-pandoc.

@bdarcus this looks to be already somewhat implemented in a pandoc filter, I guess I need to look at what's required to allow this filter to work also for ox-pandoc. In general I want to avoid adding features here and rather aim to make sure whatever clever stuff is done on the Org or pandoc side works properly.

EmmanuelCharpentier commented 2 years ago

Bruce :

To clarify, that had nothing to do with your request; I just meant pandoc converted markdown citations to org-cite citations correctly.

I misread you. I thought you solved the problem I have...

Alex :

Please feel free to open a separate issue for this and point to your comments here.

I'll certainly do that.

But thinking of it makes me think that we should involve also the Org mailing list. : we need there some form of handling of Zotero libraries (i. e. some processing of #+bibliography:), of Zotero pointers (i. e. processing by [cite:...]) and reference list generation (i. e. #+print_bibliography).

Since Org tries to maintain some orthogonality between text structure and export, it may be necessary to create some on-the-fly conversions between bibliography formats. Not trivial...

I may post (after due thinking...) something on the Org mailing list, Cc'ing a new issue on this github repository. Is that a convenient way of cross-communication ?

Thoughts ?

bdarcus commented 2 years ago

Really the only way I see of adding support for the feature via org is through the oc-csl processor, which really means citeproc-el. Though even if that were to happen, I am guessing some supporting code in the ODT export machinery would help here.

cc @andras-simonyi, in case he has thoughts.

andras-simonyi commented 2 years ago

Really the only way I see of adding support for the feature via org is through the oc-csl processor, which really means citeproc-el.

Actually, I'm working towards releasing version 1.0 of citeproc-el (...) and one of the relatively few things on my to-do list is improving odt output, so I could look into this as well. An issue about this would be useful, especially with a description of (or reference[s] about) the expected output. One thing which might or might not be a problem is that oc-csl now supports far more citation styles than Zotero IIUC.

Though even if that were to happen, I am guessing some supporting code in the ODT export machinery would help here.

yes, unfortunately, this type of change can easily require modifying other odt-related parts, e.g., the default template.

emacsorphanage / ox-pandoc

Correctly support org-cite functionality #1