search

kevlened / pytest-parallel

A pytest plugin for parallel and concurrent testing
https://github.com/browsertron/pytest-parallel/issues/104#issuecomment-1293941066
MIT License
313 stars 60 forks source link

AttributeError: Can't pickle local object on Windows 10 #74

Open esterity opened 4 years ago

esterity commented 4 years ago

Python 3.8.3, pytest-parallel 0.1.0 enviroment on Windows 10 pytest --tests-per-worker 4

It works correctly with pytest-parallel 0.0.10 There isn't have ForkingPickler on Windows, as far as I know. 

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\_pytest\main.py", line 191, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\_pytest\main.py", line 247, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "c:\projects\pytestparallel\venv\lib\site-packages\pytest_parallel\__init__.py", line 302, in pytest_runtestloop
INTERNALERROR>     process.start()
INTERNALERROR>   File "C:\Users\a.bobrov\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\process.py", line 112, in start
INTERNALERROR>     self._popen = self._Popen(self)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 223, in _Popen
INTERNALERROR>     return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\context.py", line 322, in _Popen
INTERNALERROR>     return Popen(process_obj)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\popen_spawn_win32.py", line 65, in __init__
INTERNALERROR>     reduction.dump(process_obj, to_child)
INTERNALERROR>   File "C:\Users\me\AppData\Local\Programs\Python\Python37-32\lib\multiprocessing\reduction.py", line 60, in dump
INTERNALERROR>     ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'
roygwells commented 4 years ago

Same issue here. Same Environment also windows 10. Going to try rolling back to 0.0.10 and see if that works for me.

chennareddy08 commented 4 years ago

Same issue on windows 7 for platform win32 -- Python 3.8.3, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 -- C:\Users\user\AppData\Local\Programs\Python\Python38-32\python.exe cachedir: .pytest_cache metadata: {'Python': '3.8.3', 'Platform': 'Windows-7-6.1.7601-SP1', 'Packages': {'pytest': '5.4.3', 'py': '1.8.1', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.9.0', 'parallel': '0.1.0', 'forked': '1.1.3', 'html': '2.1.1', 'rerunfailures': '9.0'}, 'JAVA_HOME': 'C:\Program Files\Java\jre1.8.0_144'}

ceddlyburge commented 4 years ago

Same issue for me, fixed by using 0.0.10

chineseluo commented 4 years ago

The problem was not fixed at 0.10

iamtech123 commented 3 years ago

Same issue for me.

platform win32 -- Python 3.7.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 rootdir: C:\Git\python-pytest-bdd, configfile: pytest.ini plugins: bdd-4.0.2, forked-1.3.0, html-3.1.1, html-reporter-0.2.6, metadata-1.11.0, parallel-0.1.0, xdist-2.3.0
collected 7 items pytest-parallel: 1 worker (process), 2 tests per worker (threads) INTERNALERROR> Traceback (most recent call last): INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages_pytest\main.py", line 269, in wrap_session INTERNALERROR> session.exitstatus = doit(config, session) or 0 INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages_pytest\main.py", line 323, in _main INTERNALERROR> config.hook.pytest_runtestloop(session=session) INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pluggy\hooks.py", line 286, in call INTERNALERROR> return self._hookexec(self, self.get_hookimpls(), kwargs) INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pluggy\manager.py", line 93, in _hookexec INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs) INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pluggy\manager.py", line 87, in INTERNALERROR> firstresult=hook.spec.opts.get("firstresult") if hook.spec else False, INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pluggy\callers.py", line 208, in _multicall INTERNALERROR> return outcome.get_result() INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pluggy\callers.py", line 80, in get_result INTERNALERROR> raise ex[1].with_traceback(ex[2]) INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pluggy\callers.py", line 187, in _multicall INTERNALERROR> res = hook_impl.function(*args) INTERNALERROR> File "c:\users\user1.virtualenvs\python-pytest-bdd-8g8otlzx\lib\site-packages\pytest_parallel__init__.py", line 302, in pytest_runtestloop INTERNALERROR> process.start() INTERNALERROR> File "c:\python377\lib\multiprocessing\process.py", line 112, in start INTERNALERROR> self._popen = self._Popen(self) INTERNALERROR> File "c:\python377\lib\multiprocessing\context.py", line 223, in _Popen INTERNALERROR> return _default_context.get_context().Process._Popen(process_obj) INTERNALERROR> File "c:\python377\lib\multiprocessing\context.py", line 322, in _Popen INTERNALERROR> return Popen(process_obj) INTERNALERROR> File "c:\python377\lib\multiprocessing\popen_spawn_win32.py", line 89, in init INTERNALERROR> reduction.dump(process_obj, to_child) INTERNALERROR> File "c:\python377\lib\multiprocessing\reduction.py", line 60, in dump INTERNALERROR> ForkingPickler(file, protocol).dump(obj) INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.init..identity'

jackdewinter commented 2 years ago

any progress on this?

strongholder commented 2 years ago

I'm also experiencing this issue.

Louis-me commented 2 years ago

Same issue for me.

    pytest.main(['%s' %path, '--workers=1', '--tests-per-worker=3', '--html=report.html','--self-contained-html', '--capture=sys'])

============================= test session starts =============================
platform win32 -- Python 3.7.9, pytest-7.1.1, pluggy-0.13.1
rootdir: D:\project\auto_web_ui
plugins: cov-2.11.1, forked-1.4.0, html-3.1.1, parallel-0.1.1, xdist-2.5.0
collected 25 items
pytest-parallel: 1 worker (process), 3 tests per worker (threads)
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\_pytest\main.py", line 268, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\_pytest\main.py", line 322, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pluggy\hooks.py", line 286, in __call__
INTERNALERROR>     return self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pluggy\manager.py", line 93, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pluggy\manager.py", line 87, in <lambda>
INTERNALERROR>     firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pluggy\callers.py", line 208, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pluggy\callers.py", line 80, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pluggy\callers.py", line 187, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "D:\app\Python37\lib\site-packages\pytest_parallel\__init__.py", line 313, in pytest_runtestloop
INTERNALERROR>     process.start()
INTERNALERROR>   File "D:\app\Python37\lib\multiprocessing\process.py", line 112, in start
INTERNALERROR>     self._popen = self._Popen(self)
INTERNALERROR>   File "D:\app\Python37\lib\multiprocessing\context.py", line 223, in _Popen
INTERNALERROR>     return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR>   File "D:\app\Python37\lib\multiprocessing\context.py", line 322, in _Popen
INTERNALERROR>     return Popen(process_obj)
INTERNALERROR>   File "D:\app\Python37\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
INTERNALERROR>     reduction.dump(process_obj, to_child)
INTERNALERROR>   File "D:\app\Python37\lib\multiprocessing\reduction.py", line 60, in dump
INTERNALERROR>     ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'
RoCCoCo13 commented 1 year ago

Hi there,

I'm experiencing the same issue. My setup versions:

collected 56 items / 54 deselected / 2 selected
pytest-parallel: 1 worker (process), 1 test per worker (thread)
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\site-packages\_pytest\main.py", line 270, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\site-packages\_pytest\main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Roaming\Python\Python310\site-packages\pluggy\_hooks.py", line 265, in __call__
INTERNALERROR>     return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Roaming\Python\Python310\site-packages\pluggy\_manager.py", line 80, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Roaming\Python\Python310\site-packages\pluggy\_callers.py", line 60, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Roaming\Python\Python310\site-packages\pluggy\_result.py", line 60, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Roaming\Python\Python310\site-packages\pluggy\_callers.py", line 39, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\site-packages\pytest_parallel\__init__.py", line 313, in pytest_runtestloop
INTERNALERROR>     process.start()
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\multiprocessing\process.py", line 121, in start
INTERNALERROR>     self._popen = self._Popen(self)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 224, in _Popen
INTERNALERROR>     return _default_context.get_context().Process._Popen(process_obj)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\multiprocessing\context.py", line 327, in _Popen
INTERNALERROR>     return Popen(process_obj)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\multiprocessing\popen_spawn_win32.py", line 93, in __init__
INTERNALERROR>     reduction.dump(process_obj, to_child)
INTERNALERROR>   File "C:\Users\alejandro.mendez\AppData\Local\Programs\Python\Python310\lib\multiprocessing\reduction.py", line 60, in dump
INTERNALERROR>     ForkingPickler(file, protocol).dump(obj)
INTERNALERROR> AttributeError: Can't pickle local object 'ArgumentParser.__init__.<locals>.identity'