UTF-8-encoded sources do not work when locale is "C"

[grothesque]

With Python 3, Sphinx does not accept UTF-8-encoded source files, when the locale is set to "C". This happens although the source_encoding configuration variable is left at its default value (See http://sphinx-doc.org/config.html#confval-source_encoding).

This behavior is problematic. For example, Debian packages get built with a "C" locale when an isolated build enviroment (cowbuilder) is used.

To reproduce the problem execute for example the following shell commands

$ wget https://pypi.python.org/packages/source/k/kwant/kwant-1.2.2.tar.gz
$ tar xzf kwant-1.2.2.tar.gz 
$ cd kwant-1.2.2 
$ python3 setup.py build
$ cd doc 
$ LC_ALL=C make html

The last command fails with the following message

Running Sphinx v1.2.3
/usr/lib/python3/dist-packages/sphinx/util/pycompat.py:83: DeprecationWarning: 'U' mode is deprecated
  f = open(filepath, 'rbU')
loading pickled environment... failed: [Errno 2] No such file or directory: '/tmp/kwant-1.2.2/doc/build/doctrees/environment.pickle'
[autosummary] generating autosummary for: index.rst, pre/about.rst, pre/authors.rst, pre/citing.rst, pre/contribute.rst, pre/index.rst, pre/install.rst, pre/license.rst, pre/whatsnew/0.2.rst, pre/whatsnew/1.0.rst, ..., reference/kwant.solvers.sparse.rst, reference/kwant.system.rst, tutorial/index.rst, tutorial/introduction.rst, tutorial/tutorial1.rst, tutorial/tutorial2.rst, tutorial/tutorial3.rst, tutorial/tutorial4.rst, tutorial/tutorial5.rst, tutorial/tutorial6.rst

Encoding error:
'ascii' codec can't decode byte 0xc3 in position 157: ordinal not in range(128)
The full traceback has been saved in /tmp/sphinx-err-getvmu2q.log, if you want to report the issue to the developers.

This is the full traceback:

# Sphinx version: 1.2.3
# Python version: 3.4.3+
# Docutils version: 0.12 release
# Jinja2 version: 2.8
# Loaded extensions:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/sphinx/cmdline.py", line 253, in main
    warningiserror, tags, verbosity, parallel)
  File "/usr/lib/python3/dist-packages/sphinx/application.py", line 147, in __init__
    self._init_builder(buildername)
  File "/usr/lib/python3/dist-packages/sphinx/application.py", line 209, in _init_builder
    self.emit('builder-inited')
  File "/usr/lib/python3/dist-packages/sphinx/application.py", line 408, in emit
    results.append(callback(self, *args))
  File "/usr/lib/python3/dist-packages/sphinx/ext/autosummary/__init__.py", line 548, in process_generate_options
    base_path=app.srcdir)
  File "/usr/lib/python3/dist-packages/sphinx/ext/autosummary/generate.py", line 120, in generate_autosummary_docs
    items = find_autosummary_in_files(sources)
  File "/usr/lib/python3/dist-packages/sphinx/ext/autosummary/generate.py", line 237, in find_autosummary_in_files
    lines = f.read().splitlines()
  File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 157: ordinal not in range(128)

[tk0miya]

I reproduce the problem with 1.2.3 and succeeded with 1.3.3. It seems already fixed at between 1.2.3 and 1.3.3. So I close this issue.

Could you upgrade your sphinx to newest one?

Thank you for reporting.

[grothesque]

Takeshi KOMIYA wrote:

I reproduce the problem with 1.2.3 and succeeded with 1.3.3. It seems already fixed at between 1.2.3 and 1.3.3. So I close this issue.

Indeed. I was using 1.2 because of a bug in 1.3.1 that has been fixed by now.

Could you upgrade your sphinx to newest one?

WIth 1.3.3 it works indeed, thanks.