[Feature request] BibTeX / Literature in Franklin

[kellertuer]

Thanks for this nice package, it looks neat and seems easy to use so far.

Would it be possible to integrate for example Bibliography.jl to have a Publication List (maybe even literature in single pages/blog posts at the bottom)? That would be awesome. Especially a publication list would be nice to have (and my starting point to not only play with Franklin but actually switch).

[tlienart]

So Bibliography.jl doesn't work for me (it's also not registered) but here's a prototype with BibTeX.jl which should show you how things can work; this is with the sandbox template

index.md

# Test with BibTex.jl

Some text here etc

## References

{{show_refs coleridge almendro maron}}

utils.jl

using BibTeX

function ref_item(ref, infos)
    io = IOBuffer()

    author = infos["author"]
    author_last, author_first = strip.(split(author, ","))

    write(io, "<li id=\"#$ref\">")
    write(io, """$author_first $author_last, <span style="font-style:italic;">$(infos["title"])</span>, $(infos["date"]).""")
    write(io, "</li>")
    return String(take!(io))
end

function hfun_show_refs(refs)
    _, allrefs = parse_bibtex(read(joinpath("_assets", "bibex.bib"), String))
    out = IOBuffer()
    write(out, "<ul>")
    for ref in refs
        infos = get(allrefs, ref, nothing)
        isnothing(infos) && continue
        write(out, ref_item(ref, infos))
    end
    write(out, "</ul>")
    return String(take!(out))
end

and that's it. The example bib file I'm using is this one and is saved in _assets/bibex.bib

Result

Notes

• If you decide to use what I'm showing above, you'll have to add a bit of logic to remove these {...} and parse things like accents which are stored in the LaTeX format (see screenshot); this is pretty easy with a few replace in the ref_item function; though ideally these are things which should be done in the packages themselves...
• Franklin has an internal way to handle references (parsing and processing of \cite, \citet, \citep); it could be hooked up with something that can parse bib files without too much effort but I'm currently a bit disappointed by the state of these libs (though you may have had a better experience in which case a working example using one of those packages would be useful)

[tlienart]

hmm this could do the job: https://lucianolorenti.github.io/BibTeXFormat.jl/latest/ though it can't handle accents apparently; a good start nonetheless. accents: https://github.com/lucianolorenti/BibTeXFormat.jl/issues/6

[kellertuer]

Oh sorry, I wasn't aware that Bibliography.jl is not yet registered, I just saw a propotype with that and Documenter recently and since I am thinking of moving my page from Jekyll to Franklin and this is the only thing I am not yet sure about... this might be a good thing to discuss here.

So yes of course, BibTeXFormat and things are a good way, too. I will check. Concerning Accents, my usual bib-files are from BibLaTeX using UTF8, then accents should also be not a problem.

[tlienart]

Right then in your case you should be able to "just" use my solution tweaking for the desired look. You could also use BibTexFormat but I'm not sure it will help you massively

[kellertuer]

I will try that then, maybe next week maybe during JuliaCon and report back.

[j-fu]

Hi, while the above variant with BibTex.jl essentially worked (kudos for being able to parse my messy bibfile after only a few fixes!), presentation of the results in html is quite hard to achieve. OTOH if you have pandoc and pandoc-citeproc installed, you can make use of the vast library of CSL styles available, e.g. you will find there ieee.csl.

Assuming a bibliography mypapers.bib in _assets, this can be achieved in the following way: Write a "hfun" calling pandoc which writes a the html formatted bibliography to stdout:

function hfun_pub()
    read(`pandoc -f markdown+yaml_metadata_block+citations+raw_html _assets/pub.md --filter pandoc-citeproc --csl=_assets/ieee.csl --bibliography=_assets/mypapers.bib   --mathjax`,String)
end

In pub.md, you just write the yaml header with the bibtex keys (these dots etc. matter) :

---
nocite: |
 @Article1,
 @Article2,
 @Article3,
...

E.g. in publications.md you write

# Publications
{{pub}}

I had this up and running for some custom static site building, now I am a proud owner of a Franklin.jl based homepage ;-).

[kellertuer]

This looks really really nice, thanks Jürgen! I will take a closer look hopefully soon.

[tlienart]

beautiful! that's something I can add in the docs as a nice example

Btw @j-fu are you happy for me to add a link to your webpage in the README? thanks!

[j-fu]

Well after sleeping over it:

• The ultimate julian way would be to implement CSL based typesetting in Julia, which seems to be an achievable goal, given that there are already several implementations in different languages.

• The pattern to use pandoc could easily be generalized to the processing of whole markdown pages. In particular, this would allow to have in-text references. Not sure though if this should be the way forward here, but pandoc is just incredibly powerful (and thus making a point for haskell btw...)

Albeit interesting, all of this is outside my priorities, so i just mention these points here for anyone liking to pick them up...

As for linking my home page - well yes, if you think it is ok enough - I won't be that active maintaining it, tough I plan to post my scientific computing course material (julia/pluto based) there in the next couple of months.

[kellertuer]

I would contribute to both a CSL based typesetting as well as something that does pandoc-ish things with Franklin, though also for me both topics are not my main priorities so I will not start such projects but would contribute if they exist.

[tlienart]

So I had a look and it seems to me that a good working compromise is to just use citeproc-js which is heavily battle tested.

In a way this would be similar to the pandoc option that @j-fu used but the project seems (more?) active and tested, also there would then be two options, one where you just load the js library and one where you pre-render. Franklin could provide some glu code to make this seamless. In a way this is similar to the code highlighting, there could be a Julia based highlighter and in fact there are a couple but they are far from as battle tested as highlight.js or prism.

It might then appear that this is somehow too slow or not flexible enough and might seed an effort towards a Julian one but this is an entire project in itself and doesn't look trivial (many finicky details).

Ps: on the pandoc front, Franklin is now a bit too far, so there could still be a plugin where the user would say "give that bit to pandoc and plug the results here untouched" (potentially an entire page) but pandoc cannot do some stuff that Franklin does and using both would likely be more annoying than helpful :)

[adigitoleo]

If the references can be processed before building the site, DOI content negotiation could be of interest. So long as all the publications have a DOI that supports it, the plain text reference can be retrieved with e.g. curl -D - -L -H "Accept: text/x-bibliography" <doi link here> (remove -D - to hide the header info) or using a simple script that can do web requests.

[tlienart]

hmm pretty cool and definitely doable

[adigitoleo]

I'll play around with it a bit, for now I'm pasting each entry in manually. My first hunch for automation is that the output might still need some touching-up, like turning the DOI number into an actual link.

[adigitoleo]

I've come up with something that will probably suffice for me for now: https://gist.github.com/adigitoleo/52d793e5fa4a8d4787ff556fae7f9454

• Doesn't rely on bib file, you only need a TOML file with some DOI links
• Citation is retrieved from crossref (or whoever services the content negotiation for your DOI)
• Formatted responses are cached to avoid repeat requests (delete the cache file to reset things)
• Authors are put inside <strong>, title in <em>, links are hyperlinked
• Can refer to the entry by the same key used in the TOML files
• Can put arbitrary text in the in-text citation (it's just a dumb href link)

It's a bit verbose for multiple citations at the moment:

{{{citeref key1 Foo & Bar (2022)}}, {{citeref key2 Foo et al. (2022)}}

Maybe I'll get around to making it nicer at some point. Also I haven't tested it very much and it is past midnight so please check it before using. Regex might need some tweaking.

[tlienart]

Very nice! and I love the big regex in the middle, does the website for which you're using this have a public repo? in which case it might be good to link to it too so users can see a demo and potentially see your latest updates.

[adigitoleo]

Working on my first blog article now, then I'll push it as an example, with a fix for the regex which didn't work for textbooks (or anything that doesn't have a journal name). The retrieved reference text has so far been fine in most cases, occasionally I get strange things in the crossref metadata (usually something in place of missing/incorrect page number declarations). But it is easy enough to touch up the refcache. I've also made the citation macro print ??? like the default one when a ref is missing from the list...

[adigitoleo]

All of the stuff can be found in here: https://sr.ht/~adigitoleo/adigitoleo-website/sources Changed to using Zola (it's just so dang fast).

For example, utils.jl.

The article with references: https://adigitoleo.srht.site/blog/geophys/drex-tests-analytical Removed, some info migrated to Python API docs.

Website also linked on my github profile. The article is very technical and mostly for self-reference. Don't worry about the content. It did however serve as a useful test for just about all of the features I will ever need (maths, images, flexboxes, refs, footnotes...) Happy to be added to README here, although that's basically all I have there at the moment.

[lrsantos11]

Hi, while the above variant with BibTex.jl essentially worked (kudos for being able to parse my messy bibfile after only a few fixes!), presentation of the results in html is quite hard to achieve. OTOH if you have pandoc and pandoc-citeproc installed, you can make use of the vast library of CSL styles available, e.g. you will find there ieee.csl.

Assuming a bibliography mypapers.bib in _assets, this can be achieved in the following way: Write a "hfun" calling pandoc which writes a the html formatted bibliography to stdout:

function hfun_pub()
    read(`pandoc -f markdown+yaml_metadata_block+citations+raw_html _assets/pub.md --filter pandoc-citeproc --csl=_assets/ieee.csl --bibliography=_assets/mypapers.bib   --mathjax`,String)
end

In pub.md, you just write the yaml header with the bibtex keys (these dots etc. matter) :

---
nocite: |
 @Article1,
 @Article2,
 @Article3,
...

E.g. in publications.md you write

# Publications
{{pub}}

I had this up and running for some custom static site building, now I am a proud owner of a Franklin.jl based homepage ;-).

@j-fu I just contructed my publication list using your ideias. Thanks for that. I have a question: In your webpage the publications are sorted by date. Did you change the ieee.csl class to reflect this? Could you give some hint on how you did it?