Cannot handle Unicode characters with Python 2.7

[ghost]

The following snippet fails with an UnicodeEncodeError on Python 2.7:

.. command-output:: echo "U+2264 ≤ LESS-THAN OR EQUAL TO"

The error is

  File ".../sphinxcontrib-programoutput/src/sphinxcontrib/programoutput/__init__.py", line 245, in run_programs
    returncode=returncode)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2264' in position 13: ordinal not in range(128)

The internet(tm) suggested that when using pattern.format(str) both str and pattern have to have the same encoding. In the above example pattern is ASCII where as str is the above UTF-8 string. The solution is to make them both UTF-8.

Python 3 does not seem to have this problem (probably due to the unicode change of str in Python 3).

Workaround: Add

programoutput_prompt_template = u'$ {command}\n{output}'

to your Sphinx conf.py (Note the u infront of the string which marks it as UTF-8).

The commits Attached three commits

• one which adds a test for Unicode (letting even the test suite itself fail on Python 2.7, see 3rd commit)
• one which applies the workaround above
• the last one is a commit to make the test suite handle Unicode strings (otherwise it fails on f.write() in sphinxcontrib-programoutput/src/sphinxcontrib/programoutput/tests/__init__.py, line 99). However, this triggers DeprecationWarning: 'U' mode is deprecated on Python 3.6 (and maybe other 3.x).

Feel free to use them as you wish.

[ghost]

To be clear: This pull request is not directly intended for merging (due to the Deprecation warnings introduced in the 3rd commit) but giving you the patches for easier inspection.

[ghost]

Any news on this?

[jamadden]

Thanks for this PR! I've updated the fixes in it and released it to PyPI as 0.14.

[ghost]

@jamadden Thanks for taking care of this.