Arxiv compatibility

[Lukas2345]

Hi, it is currently rather cumbersome to use put documents using BibLatex on the ArXiv. Since I believe that a number of BibLatex users also frequently needs to upload documents on the ArXiv, I suggest to add a feature to simplify this.

Current situation: 1) One cannot upload PDF-Files which are generated via TeX to the ArXiv, instead one is asked to provide the source, and it is compiled on their server. 2) One cannot upload .bib-Files, instead one is asked to upload the .bbl-File 3) .bbl-Files created with different versions of BibLatex are (mostly) incompatible.

The TexLive currently used by the Arxiv requires a bbl-File in the format version 2.8, which is BibLatex 3.5, 3.6 and 3.7. Otherwise the user gets an error message "File 'blah.bbl' is wrong format version - expected 2.8". (I did not find the information to which package version this corresponds in the documentation, so I looked at the biblatex.sty for various versions.)

The ArXiv updates its TexLive rather infrequently (the current one is 2016, before it was 2011), and the included packages are not updated in the meantime. So this 2.8 is most likely to stay for a few years.

Suggestion: Add a package option to Biblatex which causes it to create a .bbl file in the format 2.8.

Optional: Ideally, this option should be ignored by the version of Biblatex used by the ArXiv. I do not know if that is possible, but this way the same tex code would compile both locally and on the ArXiv.

[plk]

This is basically impossible - the .bbl for biblatex contains custom macros defined in biblatex.sty and not generic TeX. So, the coupling is very tight and ignoring the version would make things simply not work. The version is in there to make sure the the right version of biber is used (which is what writes the .bbl). The only real solution is seeing if Arxiv can update its biblatex version or making sure you use the same biblatex version as Arxiv. If Arxiv uses TL, perhaps they can just update biblatex?

[moewew]

I don't think that the arXiv people are keen on updating their systems more often.

If an option like this were implemented the only safe way I can think of at the moment would essentially amount to including a copy of an older version of biblatex in the current version. That does not seem like a good idea.

@Lukas2345 I know of people that basically froze their TeX live at (more or less) the exact version that arXiv are running. (They copied their texmf files over to a new directory.) Since you can have several TeX lives installed at the same time, that might be a viable option. Of course it is hard to do that retroactively. But you can at least get older versions of biblatex and Biber from sourceforge.

[lenis2000]

One needs a texlive 2016 with the 3.5 version of biblatex. What worked for me, is

• I download mactex basic 2016,
• switched to that version,
• updated the tlmgr (sudo tlmgr update --self),
• installed all the needed packages (without which pdflatex chokes),
• removed the system biblatex.

Then downloaded the zipped release of biblatex-3.5 and install it into local texmf tree using obuild (for some reason cloning the whole repo and using obuild to get the appropriate version does not work!)

This produces a basic tex 2016 with a few extra packages, and a desired version of biblatex.

---

PS. If someone reproduces the issue that obuild from the current version does not install biblatex 3.5 properly, maybe we can file this as an issue

[moewew]

@lenis2000 Thanks for the comment. A few remarks:

• If you want to use all of biblatex's features, you should use the Biber backend. Then you would have to install the corresponding version of Biber as well (you can get them from sourceforge: https://sourceforge.net/projects/biblatex-biber/files/biblatex-biber/, biblatex 3.5 needs Biber 2.6).
• obuild only installs the files it can find and gives them the version number you specify, it does not actually install the release version of the version you supply it with. So if you clone the repo now and run the install script with 3.5 as version argument it will install what is essentially biblatex 3.10, but it will modify the version strings in a way that the installation identifies itself as 3.5. This is definitely not what you want. So you will need to download the 3.5 release either here using tags or from sourceforge (https://sourceforge.net/projects/biblatex/files/)
• You might have to install other packages so you can compile more complicated documents as well. Ideally you'd install the same version that is present on arXiv as well. (For many packages that is probably not crucial, either because they are stable now and not updated that frequently, or because they don't rely on external binaries.)

[beatngu13]

A little workaround: As a MacTex user, it's quite easy to handle multiple TeX Live versions: Tex Live Utility > Configure > Change Default TeX Live version… (which allows you to upgrade or downgrade to the version used by arXiv.)

[moewew]

For people coming here looking for help with the dreaded

Package biblatex Warning: File '<file>.bbl' is wrong format version - expected 2.8.

version on arXiv submissions, please refer to https://github.com/plk/biblatex/wiki/biblatex-and-the-arXiv in the Wiki.

[djsutherland]

I've also had success uploading both the locally-produced .bbl and the local version of biblatex to arXiv, with this script new link. I guess that might break on some weird cases, but it works for what I've needed it for.

(Edit: I just added that to the wiki page linked above.)

(Edit by @moewew: Added link to official repository.)

[moewew]

@dougalsutherland Thank you for the addition. Is there any place where people can report issues with your Python script? I don't know a lot about Python and I really don't want to have to tell people that we don't maintain the script, but that I have no idea where to report issues.

[moewew]

@dougalsutherland I clarified that we do not maintain the script in the Wiki (https://github.com/plk/biblatex/wiki/biblatex-and-the-arXiv/_compare/629b4424f501768e37d6f1bfba893664bd1fc688), hope you are OK with that. If you have an official bugtracker/support address for that script, we could add that to the Wiki as well.

[Spyros2015]

I’ve the same problem. I’m using Xelatex in Debian Jessy. The message is: bbl is wrong format version expected 2.9.

[moewew]

@Spyros2015 If this is about the arXiv, please see https://github.com/plk/biblatex/wiki/biblatex-and-the-arXiv. If not, it is most likely that your versions of biblatex and Biber do not match (see for example https://tex.stackexchange.com/q/286706/35864). We can't really give personalised LaTeX support or debugging advice here, this tracker is for bug reports and feature suggestions. I suggest you ask at https://latex.org/ or https://tex.stackexchange.com/ or a different LaTeX site. If you do ask a question, please make sure you have read the forum rules on good questions and include a short example document, often called MWE: https://tex.meta.stackexchange.com/q/228/35864 http://www.minimalbeispiel.de/mini-en.html, the relevant .log and .blg output as well as exact information about your system (which biblatex version do you have, which Biber version, how did you install TeX, ...) and what you did to obtain the result you get.

[YodaEmbedding]

Wouldn't it make more sense for arxiv to allow .bib source files rather than enforcing .bbl compiled files?

arxiv mentions:

We do not run BibTeX because the .bib database files can be quite large, and the only thing necessary to resolve the references for a given paper is the .bbl file.

I'm not sure I understand what "large" means here. Are authors submitting .bib files containing 100000 entries? Couldn't that be mitigated with a simple file size check?

[moewew]

It would certainly make things easier for users if the arXiv accepted .bib files instead of .bbl files, because the .bib files are not tied to a specific version of biblatex in the way .bbl files are. But it would required a very different workflow for the arXiv because they would have to run a bibliography tool (BibTeX or Biber). I can understand the reluctance to change anything with that from their part. I don't really buy the file size argument as the most important point, but it is true that .bib files can be quite large and that only a small subset is needed for a particular paper. I have no doubt that there are authors with huge databases and I have no trouble imagining that a small subset would not worry about uploading a .bib file with only the relevant entries, but would instead upload the whole thing.

[pauloney]

A lot of this hinges on a very simple problem. There are no good tools to do minimization of BIB files. They all have some form of flaw one way or another and do an incomplete job. Some need to be extended by hand to include new entry-types (like bibexport) an the solution provided by biber does not extract non-standard fields like "fjournal", "mrnumber" etc ... that are necessary for the good maintenance of the bib files downstream.

Providing a standard way to minimize the files properly -- that is -- by including the FULL bibtex record used in the main file we would eliminate one of the most common complaints of publishers.

Paulo Ney

On Mon, Aug 30, 2021 at 2:41 AM moewew @.***> wrote:

It would certainly make things easier for users if the arXiv accepted .bib files instead of .bbl files, because the .bib files are not tied to a specific version of biblatex in the way .bbl files are. But it would required a very different workflow for the arXiv because they would have to run a bibliography tool (BibTeX or Biber). I can understand the reluctance to change anything with that from their part. I don't really buy the file size argument as the most important point, but it is true that .bib files can be quite large and that only a small subset is needed for a particular paper. I have no doubt that there are authors with huge databases and I have no trouble imagining that a small subset would not worry about uploading a .bib file with only the relevant entries, but would instead upload the whole thing.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/plk/biblatex/issues/663#issuecomment-908198540, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAR7WYUK5J62CL5KO5B4ZLDT7NG3BANCNFSM4EFMFQQQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[ncihnegn]

My tool might be helpful: https://github.com/ncihnegn/bbl2bib