Bibliography lost when UnicodeEncodeError is triggered

[RMeli]

I think there is a serious problem when UnicodeEncodeError is triggered because 'latex' codec can't encode character:

UnicodeEncodeError: 'latex' codec can't encode character '\ufb03' in position 0: don't know how to translate 'ffi' into latex

The bibliography seem to be erased when it is modified in place (betterbib -i).

[nschloe]

Let me see an example bib and I'll see what I can do.

[RMeli]

$ betterbib --version
betterbib 3.5.15, Python 3.7.6 | packaged by conda-forge | (default, Mar 23
2020, 22:45:16) [Clang 9.0.1 ]

---

Bibliographic entry with character \ufb03 (named bib.tex).

 @article{Francoeur2020,
  title        = {Three-Dimensional Convolutional Neural Networks and a Cross-Docked Data Set for Structure-Based Drug Design},
  volume       = {60},
  issn         = {1549-9596, 1549-960X},
  doi          = {10.1021/acs.jcim.0c00411},
  abstractnote = {One of the main challenges in drug discovery is predicting protein− ligand binding affinity. Recently, machine learning approaches have made substantial progress on this task. However, current methods of model evaluation are overly optimistic in measuring generalization to new targets, and there does not exist a standard data set of sufficient size to compare performance between models. We present a new data set for structure-based machine learning, the CrossDocked2020 set, with 22.5 million poses of ligands docked into multiple similar binding pockets across the Protein Data Bank, and perform a comprehensive evaluation of grid-based convolutional neural network (CNN) models on this data set. We also demonstrate how the partitioning of the training data and test data can impact the results of models trained with the PDBbind data set, how performance improves by adding more lower-quality training data, and how training with docked poses imparts pose sensitivity to the predicted affinity of a complex. Our best performing model, an ensemble of five densely connected CNNs, achieves a root mean squared error of 1.42 and Pearson R of 0.612 on the affinity prediction task, an AUC of 0.956 at binding pose classification, and a 68.4% accuracy at pose selection on the CrossDocked2020 set. By providing data splits for clustered cross-validation and the raw data for the CrossDocked2020 set, we establish the first standardized data set for training machine learning models to recognize ligands in noncognate target structures while also greatly expanding the number of poses available for training. In order to facilitate community adoption of this data set for benchmarking protein−ligand binding affinity prediction, we provide our models, weights, and the CrossDocked2020 set at https:// github.com/gnina/models.},
  number       = {9},
  journal      = {Journal of Chemical Information and Modeling},
  author       = {Francoeur, Paul G. and Masuda, Tomohide and Sunseri, Jocelyn and Jia, Andrew and Iovanisci, Richard B. and Snyder, Ian and Koes, David R.},
  year         = {2020},
  month        = {Sep},
  pages        = {4200–4215}
}

---

Steps to reproduce the problem:

$ betterbib -i bib.tex
$ cat bib.tex
$

[RMeli]

@nschloe Thanks for the very fast reply. So fast, in fact, that I was still preparing and checking a minimal example to reproduce the problem. =)

[nschloe]

Fixed released, thanks for the report. This was indeed a serious issue.

[RMeli]

Amazing, thank you very much for taking care of this so quickly!