AttributeError: ("'FuturesSession' object has no attribute 'session'", 'occurred at index 0')

[luqinghui]

I want to use the ProcessPoolExecutor, but got the following error:

AttributeError: ("'FuturesSession' object has no attribute 'session'", 'occurred at index 0')

my code like this:

from requests_futures.sessions import FuturesSession
from concurrent.futures import ProcessPoolExecutor

session = FuturesSession(executor=ProcessPoolExecutor(max_workers=5))
url = "xxxx"
params = "xxxx"
f = session.post(url, data=params)
print(f.result())

my environment is macOS 10.14 and python 3.7

[ross]

Hrm. That's definitely a new one to me. Thanks for the code snippet. What version of python are you running it with? The ProcessPoolExecutor has always been a pain and it looks like 3.7 has issues that (3.4 which was the last known to be tested and is mentioned in the README) didn't. ☹️

(env3)otter:ttt ross$ ./test.py
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/private/tmp/ttt/env3/lib/python3.7/site-packages/requests_futures/sessions.py", line 65, in request
    if self.session:
AttributeError: 'FuturesSession' object has no attribute 'session'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./test.py", line 9, in <module>
    print(f.result())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
AttributeError: 'FuturesSession' object has no attribute 'session'

[luqinghui]

Thanks. I am using python3.7 created by virtualenv.

[ross]

Still not sure why, but it looks like there's a 2nd FuturesSession object (based on some debug prints)

(env3)otter:ttt ross$ ./test.py
<requests_futures.sessions.FuturesSession object at 0x1016330b8>
<requests_futures.sessions.FuturesSession object at 0x1016330b8>
<requests_futures.sessions.FuturesSession object at 0x101633ba8>
concurrent.futures.process._RemoteTraceback:
"""
Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/process.py", line 232, in _process_worker
    r = call_item.fn(*call_item.args, **call_item.kwargs)
  File "/private/tmp/ttt/env3/lib/python3.7/site-packages/requests_futures/sessions.py", line 68, in request
    if self.session:
AttributeError: 'FuturesSession' object has no attribute 'session'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "./test.py", line 9, in <module>
    print(f.result())
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
AttributeError: 'FuturesSession' object has no attribute 'session'
(env3)otter:ttt ross$

It's looking like something has changed in the way that ProcessPoolExecutor works that breaks sharing the FuturesSession object across them. That's always been a problem and at times I've just doc'd that they're not supported. Someone got them working, i guess back in the python 3.4 times. I might be able to dig in a bit more and if I'm lucky figure out what's up.

In general I'd recommend going with the default ThreadPoolWorker unless you specifically need processes for some reason.

[ross]

Another debug run, including the pid and method name:

(env3)otter:ttt ross$ ./test.py
[31287,
 '__init__',
 <requests_futures.sessions.FuturesSession object at 0x10584e0b8>]
[31287,
 'request',
 <requests_futures.sessions.FuturesSession object at 0x10584e0b8>]
[31288,
 'request',
 <requests_futures.sessions.FuturesSession object at 0x10584eba8>]
concurrent.futures.process._RemoteTraceback:
...

It's generally not yet clear to me why request is being called multiple times.

[ross]

☹️ looks like this was fixed in master, but I hadn't yet cut a new release. Just did that 0.9.8, https://pypi.org/project/requests-futures/#history.

[luqinghui]

@ross , thanks a lot, i am using ThreadPoolWorker now. I'll clone the master to try later.