sort fails when .aux file has non-ascii chars

[GoogleCodeExporter]

sort tries to use the locale defined in the enviroment, but if the file has a 
different encoding then it fails. Setting LC_ALL=C fixes, as suggested by 
sort.

Output from sort:

sort: string comparison failed: Illegal byte sequence
sort: Set LC_ALL='C' to work around the problem.
sort: The strings compared were `\\@writefile{toc}{\\contentsline 
{subsection}{\\numberline {0.1.1}Amplifica\\c c\\
~ao}{1}{subsection.0.1.1}}' and `\\newlabel{sec:bar}\347\343o\\relax 
}{subsection.0.1.1}{}}'.

There's a kpsewhich error following, which might or might not be related. I 
think it happened even when the sort succeeded (for instance, by removing 
accented chars from the file).

example.tex attached reproduces the problem.
make.out is the output of:
$ make clean && make SHELL_DEBUG=1 example.pdf

Original issue reported on code.google.com by nilton.v...@gmail.com on 11 Mar 2010 at 4:35

Attachments:

• example.tex
• make.out

[GoogleCodeExporter]

PS: You'll probably only get this error if you have LANG=en_US.UTF-8 or 
equivalent, 
since that specific latin1 encoding (in example.tex) is invalid UTF-8.

Original comment by nilton.v...@gmail.com on 11 Mar 2010 at 6:34

[GoogleCodeExporter]

I am extremely hesitant to handle this automatically.  Try the following:

Create a Makefile.ini file with this line in it:

export LC_ALL = C

You can put that file in either ~/.latex-makefile/ or in the current source 
directory.  
Let me know if that solves the problem.

I'd love to solve this issue more automatically, but I worry that we'd be using 
too 
large a hammer for it.  How, for example, does LC_ALL interact with the 
invocation of 
latex?  Until we know, I'm not sure we can apply this solution generally.

Original comment by shiblon on 11 Mar 2010 at 2:36

[GoogleCodeExporter]

Try r647a2bcc3ec6.  It has the necessary magic built in.  It can be overridden 
in the 
Makefile.ini, too.

Original comment by shiblon on 11 Mar 2010 at 2:41

• Changed state: Fixed

[GoogleCodeExporter]

Putting it in the Makefile.ini works, but it should actually be done globally. 
Latex 
certainly does not care about the locale, since any tex document should be 
reproducible exactly, provided you're using the same version (and the same 
version 
for all packages).

+   # This is to fix issues with 'sort' on systems that are using utf-8, but that
+   # have non-ascii characters in the aux file.  For some reason, specifying -i    
+   # ignore non-printable characters is not sufficient.
+   export LC_ALL       ?= C

LGTM

But I would change the comment to something like:

# Sets LC_ALL=C, by default, so that the locale-aware tools, like sort, be
# immune to changes to the locale in the user environment.

Original comment by nilton.v...@gmail.com on 11 Mar 2010 at 7:33

[GoogleCodeExporter]

I like it.  I'm changing the text and submitting.  Marking verified, too :)

Original comment by shiblon on 11 Mar 2010 at 7:48

• Changed state: Verified

[GoogleCodeExporter]

New version is uploaded as 2.2.0-beta2.  Enjoy!

Note that if you used a version 2.1.x to report this bug, the new makefile is 
going to be 
fairly disrupted to you.  It switches to pdflatex instead of latex -> dvips -> 
p2pdf.

Original comment by shiblon on 11 Mar 2010 at 7:52