Intermittent AttributeError regarding "ProtoTest"

[dougthor42]

Summary

I'm getting an error every once in a while when running green PyBank -vvv on my PyBank program.

The odd thing is that I'll run it once, get the error, and then run it again with no issue.

I'm trying to come up with a way to reproduce it, but I haven't found a reliable method yet.

In addition, sometimes the test runner hangs. When it does, i've seen it hang on two different tests:

• test_sqlite_iterdump in \tests\test_orm.py
• test_get_salt_populates_file in \tests\test_crypto.py

Pressing CTRL+C results in the following error:

.   test_get_salt_populates_file
Traceback (most recent call last):
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\WinPython34_x64\scripts\..\python-3.4.3.amd64\Scripts\green.exe\__main__.py", line 9, in <module>
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\cmdline.py", line 75, in main
    result = run(test_suite, stream, args, testing)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\runner.py", line 107, in run
    msg = queue.get()
  File "<string>", line 2, in get
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\multiprocessing\managers.py", line 732, in _callmethod
    kind, result = conn.recv()
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\multiprocessing\connection.py", line 250, in recv
    buf = self._recv_bytes()
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\multiprocessing\connection.py", line 306, in _recv_bytes
    [ov.event], False, INFINITE)
InterruptedError: [Errno 4] Interrupted function call

I'm not yet sure if this is related or not.

I've never seen the AttributeError nor the hang when running tests on Travis or Appveyor.

Versions and Environment

Python 3.4.3 64-bit WinPython 3.4.3.2 Running in Windows CMD Windows 7 Professional, SP2, 64-bit Green v2.2.0

---

What can I do to help debug this issue?

Error text:

Error in green.suite.GreenTestSuite.<green.suite.GreenTestSuite
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\process.py", line 299, in poolRunner
    test.run(result)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\suite.py", line 114, in run
    test(result)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\unittest\suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\suite.py", line 114, in run
    test(result)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\unittest\suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\suite.py", line 114, in run
    test(result)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\unittest\case.py", line 625, in __call__
    return self.run(*args, **kwds)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\unittest\case.py", line 553, in run
    result.startTest(self)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\result.py", line 241, in startTest
    self.start_callback(test)
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\site-packages\green\process.py", line 270, in start_callback
    queue.put(test)
  File "<string>", line 2, in put
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\multiprocessing\managers.py", line 747, in _callmethod
    raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:
---------------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\multiprocessing\managers.py", line 228, in serve_client
    request = recv()
  File "C:\WinPython34_x64\python-3.4.3.amd64\lib\multiprocessing\connection.py", line 251, in recv
    return ForkingPickler.loads(buf.getbuffer())
AttributeError: Can't get attribute 'ProtoTest' on <module 'green.result' from 'C:\\WinPython34_x64\\python-3.4.3.amd64\
\lib\\site-packages\\green\\result.py'>
---------------------------------------------------------------------------

Ran 45 tests in 1.776s

FAILED (errors=1, passes=43, skips=1)

[CleanCut]

Can you reproduce it when you run with just one process? (--processes 1)

[dougthor42]

It looks like it doesn't happen with --processes 1 but I still haven't been able to reliably reproduce it, so it's hard to say if the issue does indeed get fixed.

It makes sense that running with multiple processes could be the issue. The tests that cause issues both act on external resources (file or SQLite database) which are created and deleted in the setUpClass and tearDownClass methods.

How does the multi-process option work? If a single TestCase class is always run on a single process, then I don't think it should cause an issue. But if the individual tests within a TestCase class are split to different processes, then I could definitely see why it causes a problem with my tests.

[CleanCut]

Okay, so race condition bugs are typical when multiple process share resources. If I were you, I'd look into the --initializer and --finalizer options to make it so that each test worker process sets up and tears down its own SQLite database. Those options are there specifically to address this type of problem.

Or you could just stick to one worker process, and just have the tests run slower.

[dougthor42]

I'll look into those options more, but at first glance I'm not sure that they'll be ideal for my usage case - I actually don't want the SQLite database to exist for the entirety of testing.

I don't mind running with a single worker so I'll just stick with that.

Thanks for your help. I'll close this and just open a new issue if I continue to have problems.

[CleanCut]

You're welcome. I hope it's smooth sailing for you from here on out! :-)