search

numberly / appnexus-client

:snake: General purpose Python client for the AppNexus API
https://appnexus-client.readthedocs.io
MIT License
39 stars 19 forks source link

Iterating on a cursor makes unnecessary round-trips to AppNexus API #39

Closed shnups closed 5 years ago

shnups commented 5 years ago

Considering the following code snippet:

import logging

from appnexus import Campaign, connect

logging.basicConfig()
logging.getLogger("appnexus-client").setLevel(logging.DEBUG)

connect(APPNEXUS_USERNAME, APPNEXUS_PASSWORD)

cursor = Campaign.find(advertiser_id=XXXXXXXX)
results = [r for r in cursor]

The execution will log the following:

DEBUG:appnexus-client:{'Authorization': None} https://api.appnexus.com/campaign?advertiser_id=XXXXXXXXXX&start_element=0&num_elements=1 None
INFO:appnexus-client:updating token
DEBUG:appnexus-client:{'Authorization': 'authn:XXXXXXXXXX'} https://api.appnexus.com/campaign?advertiser_id=XXXXXXXXXX&start_element=0&num_elements=1 None
DEBUG:appnexus-client:{'Authorization': 'authn:XXXXXXXXXX'} https://api.appnexus.com/campaign?advertiser_id=XXXXXXXXXX&start_element=0&num_elements=100 None

Outside of the first two lines that are authentication-related, there was two round-trips to access the content of a single page on AppNexus side.

Therefore, in cases where AppNexus API doesn't handle pagination/filtering properly (ChangeLog and ChangeLogDetails services, for example) and sends the whole batch-sized page all the time, getting a single page costs us the same as getting two.

The call to cursor.count() in cursor.iter_pages() is to blame for the additional round-trip: https://github.com/numberly/appnexus-client/blob/64013debe23c03a19c186ac46683a753c49e8483/appnexus/cursor.py#L89-L95