Decode error in pytest.ini with chinese characters

[nicoddemus]

Reported originally in https://github.com/pytest-dev/pytest/issues/3799 by @edsion1107

@edsion1107 a PR here (with accompanying test) would be welcome!

---

• pytest 3.7.1

pytest.ini:

[pytest]
log_file = pytest.log
log_file_level = INFO
log_file_format = %(asctime)s %(module)s.%(funcName)s %(levelname)s %(message)s
log_cli=true
log_cli_level = WARNING
log_cli_format = %(msecs)d %(filename)s(%(lineno)d) %(levelname)s %(message)s
# 中文
;--basetemp=../results
;--tap-files
;--html=report.html --self-contained-html

I have tried to add Chinese comment ,then this error occurs:

Traceback (most recent call last):
  File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.2.1\helpers\pycharm\_jb_pytest_runner.py", line 31, in <module>
    pytest.main(args, plugins_to_load)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\__init__.py", line 55, in main
    config = _prepareconfig(args, plugins)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\__init__.py", line 180, in _prepareconfig
    pluginmanager=pluginmanager, args=args
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\pluggy\hooks.py", line 258, in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\pluggy\manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\pluggy\manager.py", line 61, in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\pluggy\callers.py", line 196, in _multicall
    gen.send(outcome)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\helpconfig.py", line 89, in pytest_cmdline_parse
    config = outcome.get_result()
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\pluggy\callers.py", line 76, in get_result
    raise ex[1].with_traceback(ex[2])
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\pluggy\callers.py", line 180, in _multicall
    res = hook_impl.function(*args)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\__init__.py", line 612, in pytest_cmdline_parse
    self.parse(args)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\__init__.py", line 777, in parse
    self._preparse(args, addopts=addopts)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\__init__.py", line 723, in _preparse
    self._initini(args)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\__init__.py", line 666, in _initini
    rootdir_cmd_arg=ns.rootdir or None,
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\findpaths.py", line 118, in determine_setup
    rootdir, inifile, inicfg = getcfg([ancestor], warnfunc=warnfunc)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\_pytest\config\findpaths.py", line 35, in getcfg
    iniconfig = py.iniconfig.IniConfig(p)
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\py\_vendored_packages\iniconfig.py", line 55, in __init__
    tokens = self._parse(iter(f))
  File "C:\Users\p_jbzhang\.virtualenvs\wecar-tMZRsXDh\lib\site-packages\py\_vendored_packages\iniconfig.py", line 83, in _parse
    for lineno, line in enumerate(line_iter):
UnicodeDecodeError: 'gbk' codec can't decode byte 0xad in position 259: illegal multibyte sequence

I tried to fix this error:

# filename: iniconfig.py
# line: 49
class IniConfig(object):
    def __init__(self, path, data=None):
        self.path = str(path)  # convenience
        if data is None:
            # f = open(self.path)     
            # add encoding params 
            f = open(self.path, encoding='utf-8')
            try:
                tokens = self._parse(iter(f))
            finally:
                f.close()
        else:
            tokens = self._parse(data.splitlines(True))

[RonnyPfannschmidt]

currently iniconfog is NOT unicode aware ^^ see #4

[RonnyPfannschmidt]

now that pytest 5 is out i believe we can support unicode markers in some sense and default to utf8 or BOM detection

[RobertT93]

i tried proposed fix locally and it seams to work f = open(self.path, encoding='utf-8')

[RonnyPfannschmidt]

closing as likely solved by #49