dawoudt / JustWatchAPI

Python 3 JustWatch.com API - https://justwatch.com
MIT License
324 stars 44 forks source link

each request opens a new session #15

Closed draogn closed 6 years ago

draogn commented 6 years ago

This is a bit of an issue, which I've only picked up on following having my ip blocked when hitting a different website :sob: Easy enough to fix - just have to change requests to requests.Session(). Will post sample logs once I get a chance.

draogn commented 6 years ago

Here's a comparison with logging turned on between using and not using requests.Session()

from justwatch import JustWatch
just_watch = JustWatch(country='GB', use_sessions=False)

Gives for a couple of calls

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None

Changing to

from justwatch import JustWatch
just_watch = JustWatch(country='GB', use_sessions=True)

Gives only one connection up front for the same calls

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None

In terms of metrics searching for 20 cases was taking around 4.2 seconds with use_sessions=False and 3.2 seconds with use_sessions=True