[easy] html report raises UnicodeEncodeError on unicode docstrings

[pylint-bot]

Originally reported by: Bjorn Pettersen (BitBucket: thebjorn, GitHub: @thebjorn?)

---

This is triggered by unicode docstrings in duplicated code, e.g. create a module where this code is duplicated:

#!python

# -*- coding: utf-8 -*-

def funcname(hello):
    u'æøå'
    for i in range(hello):
        print 'world'
    return hello + 'world'

and you'll get the following error (notice that there is no reference to which file is in error):

#!python

(dev) go|c:\srv\tmp> pylint upyli -fhtml
No config file found, using default configuration
Traceback (most recent call last):
  File "c:\python27\Lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "c:\python27\Lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "c:\srv\venv\dev\Scripts\pylint.exe\__main__.py", line 9, in <module>
  File "c:\srv\venv\dev\lib\site-packages\pylint\__init__.py", line 23, in run_pylint
    Run(sys.argv[1:])
  File "c:\srv\venv\dev\lib\site-packages\pylint\lint.py", line 1332, in __init__
    linter.check(args)
  File "c:\srv\venv\dev\lib\site-packages\pylint\lint.py", line 747, in check
    self._do_check(files_or_modules)
  File "c:\srv\venv\dev\lib\site-packages\pylint\lint.py", line 878, in _do_check
    checker.close()
  File "c:\srv\venv\dev\lib\site-packages\pylint\checkers\similar.py", line 320, in close
    self.add_message('R0801', args=(len(couples), '\n'.join(msg)))
  File "c:\srv\venv\dev\lib\site-packages\pylint\checkers\__init__.py", line 101, in add_message
    self.linter.add_message(msg_id, line, node, args, confidence)
  File "c:\srv\venv\dev\lib\site-packages\pylint\utils.py", line 410, in add_message
    (abspath, path, module, obj, line or 1, col_offset or 0), msg, confidence))
  File "c:\srv\venv\dev\lib\site-packages\pylint\reporters\html.py", line 70, in handle_message
    self.msgs += [str(getattr(msg, field)) for field in self.msgargs]
UnicodeEncodeError: 'ascii' codec can't encode characters in position 87-89: ordinal not in range(128)

---

• Bitbucket: https://bitbucket.org/logilab/pylint/issue/634

[pylint-bot]

Original comment by Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Cool find, thanks!

[pylint-bot]

Original comment by Fabio Natali (BitBucket: fnstudio, GitHub: @fnstudio?):

---

Hi, it doesn't seem I can reproduce it on my system. I tried both Python 2 and 3.

[pylint-bot]

Original comment by Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Neither do I. Tested on Windows and on Linux, Python 2 and Python 3. If you could provide more details how to reproduce it, that would be awesome. Also, we tried with Pylint 1.5.0 from the repository, could you test with it to see if it reproduces?

[pylint-bot]

Original comment by Bjorn Pettersen (BitBucket: thebjorn, GitHub: @thebjorn?):

---

#!python

(dev) go|w:\srv\tmp> pylint --version
No config file found, using default configuration
pylint 1.4.4,
astroid 1.3.8, common 1.0.2
Python 2.7.8 (default, Jun 30 2014, 16:03:49) [MSC v.1500 32 bit (Intel)]

(dev) go|w:\srv\tmp> python -V
Python 2.7.8

(dev) go|w:\srv\tmp> tree pylint634
pylint634
|   `-- __init__.py
|   `-- hello1.py
    `-- hello2.py

0 directories, 3 files

(dev) go|w:\srv\tmp> cat pylint634\hello1.py
# -*- coding: utf-8 -*-

def funcname(hello):
    u'A▌A,A¥'
    print 'must'
    print 'have'
    print 'enough'
    print 'lines'
    print 'to trigger'
    for i in range(hello):
        print 'world'
    return str(hello) + 'world'

(dev) go|w:\srv\tmp> cat pylint634\hello2.py
# -*- coding: utf-8 -*-

def funcname2(n):
    u'A▌A,A¥'
    print 'must'
    print 'have'
    print 'enough'
    print 'lines'
    print 'to trigger'
    for i in range(n):
        print "world"
    return str(n) + "world"

(dev) go|w:\srv\tmp> pylint pylint634 -fhtml
No config file found, using default configuration
Traceback (most recent call last):
  File "C:\Python27\Lib\runpy.py", line 162, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "C:\Python27\Lib\runpy.py", line 72, in _run_code
    exec code in run_globals
  File "w:\srv\venv\dev\Scripts\pylint.exe\__main__.py", line 9, in <module>
  File "w:\srv\venv\dev\lib\site-packages\pylint\__init__.py", line 23, in run_pylint
    Run(sys.argv[1:])
  File "w:\srv\venv\dev\lib\site-packages\pylint\lint.py", line 1332, in __init__
    linter.check(args)
  File "w:\srv\venv\dev\lib\site-packages\pylint\lint.py", line 747, in check
    self._do_check(files_or_modules)
  File "w:\srv\venv\dev\lib\site-packages\pylint\lint.py", line 878, in _do_check
    checker.close()
  File "w:\srv\venv\dev\lib\site-packages\pylint\checkers\similar.py", line 320, in close
    self.add_message('R0801', args=(len(couples), '\n'.join(msg)))
  File "w:\srv\venv\dev\lib\site-packages\pylint\checkers\__init__.py", line 101, in add_message
    self.linter.add_message(msg_id, line, node, args, confidence)
  File "w:\srv\venv\dev\lib\site-packages\pylint\utils.py", line 410, in add_message
    (abspath, path, module, obj, line or 1, col_offset or 0), msg, confidence))
  File "w:\srv\venv\dev\lib\site-packages\pylint\reporters\html.py", line 70, in handle_message
    self.msgs += [str(getattr(msg, field)) for field in self.msgargs]
UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-75: ordinal not in range(128)

(dev) go|w:\srv\tmp>

[pylint-bot]

Original comment by Bjorn Pettersen (BitBucket: thebjorn, GitHub: @thebjorn?):

---

(you can also create a pylintrc files and set the min-similarity-lines lower..)

[pylint-bot]

Original comment by Bjorn Pettersen (BitBucket: thebjorn, GitHub: @thebjorn?):

---

I can test with 1.5.0, but I can't seem to install it cleanly.. In a fresh virtualenv I pip-installed astroid and logilab-common, then did hg clone https://bitbucket.org/logilab/pylint followed by python setup.py install. When I run pylint I get:

#!python

(pylint634) go|w:\srv\tmp> pylint pylint634 -fhtml
Problem importing module base.py: cannot import name helpers
Problem importing module base.pyc: cannot import name helpers
Problem importing module classes.py: cannot import name InconsistentMroError
...

(pylint634) w:\srv\venv\pylint634>pip freeze
You are using pip version 6.0.6, however version 7.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
astroid==1.3.8
colorama==0.3.3
logilab-common==1.0.2
pylint==1.5.0
six==1.9.0

[pylint-bot]

Original comment by Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

This example is much better, thanks. I can reproduce it.

[pylint-bot]

Original comment by Claudiu Popa (BitBucket: PCManticore, GitHub: @PCManticore):

---

Oh, the problem is that you have to install astroid 1.4.0 first.

[pylint-bot]

Original comment by Bjorn Pettersen (BitBucket: thebjorn, GitHub: @thebjorn?):

---

Ah, that made everything run better :-) Same UnicodeEncodeError with 1.5.0 though.