ross / requests-futures

Asynchronous Python HTTP Requests for Humans using Futures
Other
2.11k stars 152 forks source link

Adding attribute to response via hook (as seen in examples) fails #105

Closed berzi closed 3 years ago

berzi commented 3 years ago

I'm trying to add an attribute to responses following the examples in the readme, approximately like so:

def get_translation(resp, *args, **kwargs):
    # ... parse response
    resp.translation = result_of_parsing

session = FuturesSession()

# NOTE: linters complain about the type of get_translation
session.hooks["response"].append(get_translation)
# I also tried:
# session.hooks["response"] = get_translation
# and linters also complain about this. Type annotations should probably be fixed?

# I'm using a dict to retain the order of the items.
# segmented is of the form dict[int, str]
requests = {}
for idx, word in segmented.items():
    future = session.get(BASE_URL + word)
    requests[idx] = future

for idx, response in requests.items():
    requests[idx] = response.result()

But when I later try to access the attribute from requests[idx].translation I get AttributeError: 'Response' object has no attribute 'translation'. I've put a debug print inside the hook to make sure it was getting called, and it is, so why does the object returned by .result() not contain this attribute I added? Note that this also happens if I use setattr() on the response instead.

I believe I have followed the examples in the readme correctly, so it is something different in my situation or are the examples no longer up to date?

ross commented 3 years ago

If I turn your example into runnable code and add a loop printing the new attribute it appears to work:

#!/usr/bin/env python

from requests_futures.sessions import FuturesSession
from pprint import pprint

def get_translation(resp, *args, **kwargs):
    resp.translation = 42

session = FuturesSession()

session.hooks["response"].append(get_translation)

segmented = {
    0: 'zero',
    1: 'one',
    2: 'two',
}
requests = {}
for idx, word in segmented.items():
    future = session.get(f'https://nghttp2.org/httpbin/?idx={idx}')
    requests[idx] = future
pprint(requests)

for idx, response in requests.items():
    requests[idx] = response.result()
pprint(requests)

for idx, response in requests.items():
    pprint([idx, response.translation])
(env) coho:tmp ross$ ./example.py
{0: <Future at 0x10203a5b0 state=running>,
 1: <Future at 0x10203a8b0 state=running>,
 2: <Future at 0x10203af70 state=running>}
{0: <Response [200]>, 1: <Response [200]>, 2: <Response [200]>}
[0, 42]
[1, 42]
[2, 42]

I'd recommend the following tweaks just to make the code read more clearly. it took me reading it a couple times to figure out what things were.

#!/usr/bin/env python

from requests_futures.sessions import FuturesSession
from pprint import pprint

def get_translation(resp, *args, **kwargs):
    resp.translation = 42

session = FuturesSession()

session.hooks["response"].append(get_translation)

segmented = {
    0: 'zero',
    1: 'one',
    2: 'two',
}
# At this we're going to make requests and hold onto the futures
futures = {}
for idx, word in segmented.items():
    future = session.get(f'https://nghttp2.org/httpbin/?idx={idx}')
    futures[idx] = future
pprint(futures)

# At this stage we have futures and we want to get the responses
responses = {}
for idx, future in futures.items():
    responses[idx] = future.result()
pprint(responses)

# Now we'll iterate over the responses accessing the "translations"
for idx, response in responses.items():
    pprint([idx, response.translation])

My guess is that you don't have what you think you have in the dict in your real code and you corrected it in the translation to this issue. Naming things more clearly will probably help that.