Open martinpengellyphillips opened 7 years ago
vitalk
commented
7 years ago Hi @martinpengellyphillips!
Thank you for the report. Sadly I have no possibility to reproduce or test your issue, but looks like it can be easily fixed. Please, test against 54-pickle-error-on-windows branch.
martinpengellyphillips
commented
7 years ago Hi @vitalk - thanks for the quick response. I've tried that branch now and it fixes the previous error. However, a new error is given now:
request = <SubRequest '_push_request_context' for <Function 'test_get_url'>>
@pytest.fixture(autouse=True)
def _push_request_context(request):
"""During tests execution request context has been pushed, e.g. `url_for`,
`session`, etc. can be used in tests as is::
def test_app(app, client):
assert client.get(url_for('myview')).status_code == 200
"""
if 'app' not in request.fixturenames:
return
app = request.getfuncargvalue('app')
# Get application bound to the live server if ``live_server`` fixture
# is applyed. Live server application has an explicit ``SERVER_NAME``,
# so ``url_for`` function generates a complete URL for endpoint which
# includes application port as well.
if 'live_server' in request.fixturenames:
> app = request.getfuncargvalue('live_server').app
..\..\..\..\python-virtualenv\shadow\Lib\site-packages\pytest_flask\plugin.py:85:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\..\..\python-virtualenv\shadow\Lib\site-packages\pytest_flask\fixtures.py:130: in live_server
server.start()
..\..\..\..\python-virtualenv\shadow\Lib\site-packages\pytest_flask\fixtures.py:67: in start
self._process.start()
c:\python27\Lib\multiprocessing\process.py:130: in start
self._popen = Popen(self)
c:\python27\Lib\multiprocessing\forking.py:277: in __init__
dump(process_obj, to_child, HIGHEST_PROTOCOL)
c:\python27\Lib\multiprocessing\forking.py:199: in dump
ForkingPickler(file, protocol).dump(obj)
c:\python27\Lib\pickle.py:224: in dump
self.save(obj)
c:\python27\Lib\pickle.py:331: in save
self.save_reduce(obj=obj, *rv)
c:\python27\Lib\pickle.py:425: in save_reduce
save(state)
c:\python27\Lib\pickle.py:286: in save
f(self, obj) # Call unbound method with explicit self
c:\python27\Lib\pickle.py:655: in save_dict
self._batch_setitems(obj.iteritems())
c:\python27\Lib\pickle.py:687: in _batch_setitems
save(v)
c:\python27\Lib\pickle.py:286: in save
f(self, obj) # Call unbound method with explicit self
c:\python27\Lib\pickle.py:554: in save_tuple
save(element)
c:\python27\Lib\pickle.py:331: in save
self.save_reduce(obj=obj, *rv)
c:\python27\Lib\pickle.py:425: in save_reduce
save(state)
c:\python27\Lib\pickle.py:286: in save
f(self, obj) # Call unbound method with explicit self
c:\python27\Lib\pickle.py:655: in save_dict
self._batch_setitems(obj.iteritems())
c:\python27\Lib\pickle.py:687: in _batch_setitems
save(v)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <multiprocessing.forking.ForkingPickler instance at 0x00000000075EA588>
obj = <thread.lock object at 0x00000000071A1130>
def save(self, obj):
# Check for persistent id (defined by a subclass)
pid = self.persistent_id(obj)
if pid is not None:
self.save_pers(pid)
return
# Check the memo
x = self.memo.get(id(obj))
if x:
self.write(self.get(x[0]))
return
# Check the type dispatch table
t = type(obj)
f = self.dispatch.get(t)
if f:
f(self, obj) # Call unbound method with explicit self
return
# Check copy_reg.dispatch_table
reduce = dispatch_table.get(t)
if reduce:
rv = reduce(obj)
else:
# Check for a class with a custom metaclass; treat as regular class
try:
issc = issubclass(t, TypeType)
except TypeError: # t is not a class (old Boost; see SF #502085)
issc = 0
if issc:
self.save_global(obj)
return
# Check for a __reduce_ex__ method, fall back to __reduce__
reduce = getattr(obj, "__reduce_ex__", None)
if reduce:
> rv = reduce(self.proto)
E TypeError: can't pickle thread.lock objects I‘m sorry, but I‘m not sure I can be of much help (I don‘t run windows), but if you come up with a solution (or someone else), I‘ll happily review and merge.
martinpengellyphillips
commented
7 years ago No worries, thanks for taking a look.
Nagasaki45
commented
7 years ago I don't know if another report of the issue will help in anyway, but here it is:
============================= test session starts =============================
platform win32 -- Python 3.6.1, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
rootdir: C:\Users\tgurion\unity\Unsocial VR\backchannel, inifile:
plugins: flask-0.10.0
collected 1 item
test_server.py E
=================================== ERRORS ====================================
_____________ ERROR at setup of test_server_not_completely_broken _____________
request = <SubRequest '_push_request_context' for <Function 'test_server_not_completely_broken'>>
@pytest.fixture(autouse=True)
def _push_request_context(request):
"""During tests execution request context has been pushed, e.g. `url_for`,
`session`, etc. can be used in tests as is::
def test_app(app, client):
assert client.get(url_for('myview')).status_code == 200
"""
if 'app' not in request.fixturenames:
return
app = request.getfuncargvalue('app')
# Get application bound to the live server if ``live_server`` fixture
# is applyed. Live server application has an explicit ``SERVER_NAME``,
# so ``url_for`` function generates a complete URL for endpoint which
# includes application port as well.
if 'live_server' in request.fixturenames:
> app = request.getfuncargvalue('live_server').app
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\site-packages\pytest_flask\plugin.py:85:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\site-packages\pytest_flask\fixtures.py:127: in live_server
server.start()
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\site-packages\pytest_flask\fixtures.py:64: in start
self._process.start()
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\multiprocessing\process.py:105: in start
self._popen = self._Popen(self)
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\multiprocessing\context.py:223: in _Popen
return _default_context.get_context().Process._Popen(process_obj)
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\multiprocessing\context.py:322: in _Popen
return Popen(process_obj)
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\multiprocessing\popen_spawn_win32.py:65: in __init__
reduction.dump(process_obj, to_child)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = <Process(Process-1, initial)>, file = <_io.BufferedWriter name=8>
protocol = None
def dump(obj, file, protocol=None):
'''Replacement for pickle.dump() using ForkingPickler.'''
> ForkingPickler(file, protocol).dump(obj)
E AttributeError: Can't pickle local object 'LiveServer.start.<locals>.<lambda>'
..\..\..\AppData\Local\Continuum\Anaconda3\envs\backchannel\lib\multiprocessing\reduction.py:60: AttributeError
=========================== 1 error in 0.89 seconds ===========================
Glazz87
commented
7 years ago @Nagasaki45 @vitalk Same problem on Windows 7 64 bit, python 3.5.3. Very sad because fixture looks very useful...
(venv) C:\Git\GitHub\pytest-flask>pytest -v tests
============================= test session starts =============================
platform win32 -- Python 3.5.3, pytest-3.1.3, py-1.4.34, pluggy-0.4.0 -- c:\git\git
hub\pytest-flask\venv\scripts\python.exe
cachedir: .cache
rootdir: C:\Git\GitHub\pytest-flask, inifile: tox.ini
plugins: flask-0.10.0
collected 22 items
tests/test_fixtures.py::TestFixtures::test_config_access PASSED
tests/test_fixtures.py::TestFixtures::test_client PASSED
tests/test_fixtures.py::TestFixtures::test_accept_json PASSED
tests/test_fixtures.py::TestFixtures::test_accept_jsonp PASSED
tests/test_fixtures.py::TestFixtures::test_request_ctx PASSED
tests/test_fixtures.py::TestFixtures::test_request_ctx_is_kept_around PASSED
tests/test_fixtures.py::TestJSONResponse::test_json_response PASSED
tests/test_fixtures.py::TestJSONResponse::test_dont_rewrite_existing_implementation
PASSED
tests/test_fixtures.py::TestClientClass::test_client_attribute PASSED
tests/test_live_server.py::TestLiveServer::test_server_is_alive ERROR
tests/test_live_server.py::TestLiveServer::test_server_url ERROR
tests/test_live_server.py::TestLiveServer::test_server_listening ERROR
tests/test_live_server.py::TestLiveServer::test_url_for ERROR
tests/test_live_server.py::TestLiveServer::test_set_application_server_name ERROR
tests/test_live_server.py::TestLiveServer::test_rewrite_application_server_name ERR
OR
tests/test_live_server.py::TestLiveServer::test_prevent_starting_live_server PASSED
tests/test_live_server.py::TestLiveServer::test_start_live_server FAILED
tests/test_live_server.py::TestLiveServer::test_add_endpoint_to_live_server FAILED
tests/test_live_server.py::TestLiveServer::test_concurrent_requests_to_live_server
FAILED
tests/test_markers.py::TestOptionMarker::test_not_debug_app PASSED
tests/test_markers.py::TestOptionMarker::test_update_application_config PASSED
tests/test_markers.py::TestOptionMarker::test_application_config_teardown PASSED
=================================== ERRORS ====================================
____________ ERROR at setup of TestLiveServer.test_server_is_alive ____________
request = <SubRequest '_push_request_context' for <Function 'test_server_is_alive'>
>
@pytest.fixture(autouse=True)
def _push_request_context(request):
"""During tests execution request context has been pushed, e.g. `url_for`,
`session`, etc. can be used in tests as is::
def test_app(app, client):
assert client.get(url_for('myview')).status_code == 200
"""
if 'app' not in request.fixturenames:
return
app = request.getfuncargvalue('app')
# Get application bound to the live server if ``live_server`` fixture
# is applyed. Live server application has an explicit ``SERVER_NAME``,
# so ``url_for`` function generates a complete URL for endpoint which
# includes application port as well.
if 'live_server' in request.fixturenames:
> app = request.getfuncargvalue('live_server').app
venv\lib\site-packages\pytest_flask\plugin.py:85:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
venv\lib\site-packages\pytest_flask\fixtures.py:127: in live_server
server.start()
venv\lib\site-packages\pytest_flask\fixtures.py:64: in start
self._process.start()
C:\Users\a.glazkov\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\pr
ocess.py:105: in start
self._popen = self._Popen(self)
C:\Users\a.glazkov\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\co
ntext.py:212: in _Popen
return _default_context.get_context().Process._Popen(process_obj)
C:\Users\a.glazkov\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\co
ntext.py:313: in _Popen
return Popen(process_obj)
C:\Users\a.glazkov\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\po
pen_spawn_win32.py:66: in __init__
reduction.dump(process_obj, to_child)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
obj = <Process(Process-1, initial)>, file = <_io.BufferedWriter name=8>
protocol = None
def dump(obj, file, protocol=None):
'''Replacement for pickle.dump() using ForkingPickler.'''
> ForkingPickler(file, protocol).dump(obj)
E AttributeError: Can't pickle local object 'LiveServer.start.<locals>.<lambd
a>'
C:\Users\a.glazkov\AppData\Local\Programs\Python\Python35-32\lib\multiprocessing\re
duction.py:59: AttributeError
Jitsusama
commented
7 years ago I'm running into the same thing. Makes me very :(
I guess I'll just roll my own.
Jitsusama
commented
7 years ago This seems to work for me fine:
import subprocess
import pytest
import requests
@pytest.fixture(scope="module", autouse=True)
def server():
server = subprocess.Popen(['python', '-m', 'app'])
time.sleep(1)
yield server
server.terminate()
def test_request():
response = requests.get('http://localhost:5000')
assert response.status_code == 200
brandonschabell
commented
5 years ago I'm running into the same issue with Windows 10 and Python 3.6.7.
psdon
commented
5 years ago Having this issue too in Windows 10 and Python 3.6
psdon
commented
5 years ago Any workaround for this?
ankostis
commented
5 years ago Improving on @Jitsusama workaround, using flask run command to launch app to properly configure, and moved server-launch in ./tests/conftest.py:
import pytest
import socket
import subprocess
@pytest.fixture(scope="session")
def flask_port():
## Ask OS for a free port.
#
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
s.bind(("", 0))
addr = s.getsockname()
port = addr[1]
return port
@pytest.fixture(scope="session", autouse=True)
def live_server(flask_port):
env = os.environ.copy()
env["FLASK_APP"] = "your_package.app_module"
server = subprocess.Popen(['flask', 'run', '--port', str(flask_port)], env=env)
try:
yield server
finally:
server.terminate()And then write your TestCases in test-files (e.g. ./tests/test_my_app.py) like this:
import pytest
import requests
def test_request(flask_port):
response = requests.get(f"http://localhost:{flask_port}"')
assert response.status_code == 200
VCleemput
commented
8 months ago Hey, I'm new here. I get the same error on Windows 10, Python 3.12 trying to set up the live-server for use with selenium. I've been looking arround, and found something about pickle not being able to serialize enough different types. A workaround using "dill" was suggested (https://stackoverflow.com/questions/73570416/python-multiprocessing-cant-pickle-local-object#comment129918523_73570416), for which you can use the "multiprocess" fork of multiprocessing (https://pypi.org/project/multiprocess). I'm going to toy arround with that, but I don't have much experience on the matter...
Not sure if Windows is supposed to be supported, but for me it fails on Windows 10 Python 2.7.12 with the following error when live_server fixture used: