JuliaDocs / Documenter.jl

A documentation generator for Julia.
https://documenter.juliadocs.org
MIT License
814 stars 480 forks source link

Proposal to add support for BibTeX citations and references #1162

Closed ali-ramadhan closed 1 year ago

ali-ramadhan commented 5 years ago

Hi Documenter devs,

A few of us would love to see support for in-text citations and reference generation in Documenter through BiBTeX (see https://github.com/climate-machine/CLIMA/issues/152 and https://github.com/climate-machine/Oceananigans.jl/pull/474).

I guess I envision being able to pass a BibTeX file to Documenter and automatically be able to cite references in the text with something like [<label>](@ref) (which would pull the info from the BibTeX file), and also generate a bibliography from the BibTex file through something like

```@bibliography


I'm wondering if this is something that could be worked on by someone unfamiliar with Documenter. Thought it might be good to open an issue first to see if it's possible and if I'm missing anything. Not sure of the best approach but here's what I've gathered so far.

It looks like there was an attempt back in 2017 by @lucianolorenti to add a `bibtex` option to `makedocs` that allowed citations with `[<label>](@ref)`: https://github.com/JuliaDocs/Documenter.jl/issues/379#issuecomment-292157714

But it had to use the Python package `pybtex` to parse bibtex files. But since then there's been a native Julia parser: https://github.com/JuliaTeX/BibTeX.jl

So maybe if we can get that `bibtex` option to work with BibTeX.jl then we can pretty easily have citations in Documenter?

But then I came across a recent issue (https://github.com/JuliaDocs/Documenter.jl/issues/1018) where @mortenpi suggested that proper bibliography support should be done via a plugin like [Citations.jl](https://github.com/adamslc/Citations.jl), although it was mentioned on Slack that it's an unmaintained package that would need some work to bring it up to speed with the current version of Documenter and it would have to be registered.

It seems like the best approach would be to polish off Citations.jl to work with Documenter v0.24 and register it? Would https://github.com/JuliaDocs/Documenter.jl/issues/745 have to be resolved first?

cc @glwagner @charleskawczynski @simonbyrne who might be interested as well.
mortenpi commented 5 years ago

Yes, I do think a plugin package is a way to go and starting from Citations is probably a good start. Among other things, it would be nice if core Documenter wouldn't depend on things like BibTeX.jl.

I won't have time to work on this feature myself really, I would be happy to brainstorm this here or on Slack. I can give a hand with whatever refactoring might be necessary for Documenter's internals though.

lucianolorenti commented 4 years ago

Hi! A while ago I ported most of pybtex to Julia to allow markdown, latex and plain text output from a bib file. The package is BibTeXFormat.jl I updated the code to work in Julia 1.2 and to use the new official bibtex parser. Some output can be seen in https://lucianolorenti.github.io/BibTeXFormat.jl/latest/#Markdown-example-1 Some tests still fail because I could not find a package like pylatexenc in Julia that allows me to convert between things like {\'e} to é.

The package still needs some work and I don't have much time to devote to this, but perhaps, it can be useful.

simonbyrne commented 4 years ago

I could not find a package like pylatexenc in Julia that allows me to convert between things like {\'e} to é.

Sounds like a great idea for a GSOC project?

simonbyrne commented 4 years ago

Or possibly a Code-In project, if someone was willing to provide some guidance?

fredrikekre commented 4 years ago

Maybe it exists in Latexify.jl? If not maybe the functionality would fit there? Not sure.

ali-ramadhan commented 4 years ago

Sorry to let this issue sit around for so long but thanks to @Azzaare and his new Bibliography.jl package I think I was able to get a minimal working example of hacking in Bibliography.jl as a Documenter.jl "plugin" without too much work!

I’ve put up the code up on https://github.com/ali-ramadhan/DocumenterBibliographyTest.jl but it's a little messy (BIBLIOGRAPHY is a global const for example) so @mortenpi suggested posting here to discuss how it can be improved. I'm also not sure where this code should be placed or if it should be part of an existing package. Perhaps if it's truly a plugin then it should be it's own package?

Too lazy to set up documentation builds but here's a screenshot of a page with two references:

@article{ahu61,
   author={Arrow, Kenneth J. and Leonid Hurwicz and Hirofumi Uzawa},
   title={Constraint qualifications in maximization problems},
   journal={Naval Research Logistics Quarterly},
   volume={8},
   year=1961,
   pages={175-191},
   doi={10.1002/nav.3800080206}
}

@book{ab94,
   author = {Charalambos D. Aliprantis and Kim C. Border},
   year = {1994},
   title = {Infinite Dimensional Analysis},
   publisher = {Springer},
   address = {Berlin},
   url = {https://www.springer.com/gp/book/9783540295860}
}
# DocumenterBibliographyTest.jl documentation

Hi!

In this page I will cite an article [ahu61](@cite) and a book [ab94](@cite).

# References

```@bibliography```

image

The idea is that you read in a .bib file with Bibliography.jl (which I called const BIBLIOGRAPHY) and the new @bibliography expands to print all the references in that file formatted with URL/DOI links in an HTML description list, and each reference gets an anchor added. Then whenever you use the @cite URL syntax, it replaces the URL with an author-year citation that links to the anchor in the bibliography. So hopefully this means that the bibliography can be placed on any page and citations should work from any page as well.

Limitations: 1 .bib file only, and @bibliography prints all entries even references that were not cited (but these are minor I think). Also https://github.com/Azzaare/Bibliography.jl/issues/3.

kellertuer commented 4 years ago

Thank you for your work, this is what I was hoping for in Markdown code :)

I wouldn't mind having this within documenter directly, though it adds Bibliography.jl as a dependency.

Just for the limitations I think one of those should be resolved, I can either have one large bib-file (which is what I prefer) and a selection printed or a bib-file per page for example and printing all. Would it be possible to print only cited ones? That would be great.

edit: another possibility is of course to have the bibliography on its own page (a bibliography.md with just one line) but to the @cites then link between pages? Might be difficult, too, so I might still prefer the per page selected variant.

ali-ramadhan commented 4 years ago

@kellertuer I updated the test repo with a bibliography on its own page (references.md) and the @cite commands work on other pages which is nice. That was my use case as well: have one big .bib file and have the references listed out on a separate page with citations peppered throughout the docs.

Regarding the limitations, I don't know enough about Documenter.jl to implement printing only cited references. The @bibliography is expanded first as it is part of ExpandersPipeline, before the @cite commands are processed which are part of DocumentPipeline, which is neccessary to obtain the anchors/links for each reference. So when expanding the bibliography, we have no knowledge of what has been cited so far.

I think it shouldn't be too hard to add support for multiple bibliographies but I'm not sure which approach would be best. We can write e.g. @bibliography books.bib to expand one .bib file but then the @cite command would require access to all bibliographies. There must be a way of implementing this.

For a very first version of this plugin, allowing only one .bib file without worrying about whether the reference was cited may be reasonable just to figure out the design of the plugin before adding more features.

kellertuer commented 4 years ago

Cool, so then one of my approaches would work!

I think I prefer the one with only-cited ones, but you are right, it might b worth first checking for the plugin and its design and then (maybe with the help from more experienced Documenter programmers) extend the plugin to the two other features.

mortenpi commented 4 years ago

This is cool @ali-ramadhan! Just a few thoughts from me:

The internal APIs, that I can see, that we'd need to formalize:

mortenpi commented 1 year ago

The plugin exists now: https://github.com/JuliaDocs/DocumenterCitations.jl

kellertuer commented 1 year ago

It not only exists, it also works really great! 👍