search

plk / biblatex

biblatex is a sophisticated bibliography system for LaTeX users. It has considerably more features than traditional bibtex and supports UTF-8
505 stars 114 forks source link

[Question]: Supporting a new TeX engine (Tectonic) #1219

Open saona-raimundo opened 2 years ago

saona-raimundo commented 2 years ago

Hi All!

First of all, thank you for the great project!

There is a new TeX/LaTeX engine called Tectonic. It already works pretty well, but does not have support for .bib files yet. The discussions tectonic-typesetting/tectonic#35 and tectonic-typesetting/tectonic#866 are that we would like to add biber to the backend, but we are not really sure how to do that, therefore this question.

Just as Tectonic re-wrote XeTeX in Rust, I personally was wondering if the only way to go is to re-write biber in Rust. Some question that I have are:

Not to lose track of the final objective, the goal is the same as biblatex, but working with Tectonic engine.

plk commented 2 years ago

Interestingly, I have been looking at Rust recently and it crossed my mind about biber but this would be a major undertaking. It's not a trivial code-base and relies heavily on the very strong Unicode and collation support in Perl. I can forsee gaps that would require new crates in Rust for things like Russian/Devengari transliteration and also BibTeX file parsing.

I have thought about biber in Lua for LuaTeX but that's not trivial for the same reasons - various Lua libraries just aren't up the standard of the Perl ones we use and so a lot of extra work would be involved.

biber's inputs and ouputs are files and so an in-memory version would be tricky to do as it stands.

saona-raimundo commented 2 years ago

Thank you very much for the great answer!

About the BibTeX file parsing, I think that is already done in Rust, with various options of crates.

About the input and output being files, as far as I understand, there is no problem with reading/writing files for tectonic. They actually write the files and then delete them (you can opt in to keep intermediary files while compiling TeX).

Before suggesting a "major undertaking", I wanted to ask what a "core biber" would look like, or, first, if it makes sense to you at all. I am thinking of something that I could try my luck and see how much it is needed, to give a more informed suggestion after this.

plk commented 2 years ago

The problem is that a "core" biber isn't really something that makes much sense - most options are so interdependent that omitting some initially won't be simpler overall, it'll mean rewriting whole subsystems later. However, I think I may have a look, say, at the various .bib format parsers to see if they do what we would want - the fact that Rust is compiled means there are more options as "pure Rust" is still likely to be fast enough unlike the "pure Perl" bibtex parsers which aren't really which means we are using an old C library for this. My biggest concern is Unicode support. We rely on fairly comprehensive UCA with CLDR tailoring and from a quick look, the CLDR tailoring part isn't so well supported in most of the crates I've looked at. There are crates which use ICU but that's a whole different beast.

saona-raimundo commented 2 years ago

Thank you!! There is so much I did not know!

A quick search led me to these links:

It seems that depending on ICU library for C would be the way to go in this case, understanding that ICU support in Rust is not so good yet.

Compared to Unicode support, a god bibtex parser sounds much easier to achieve. If you do not find something that meets your needs, please let me know!