Closed draogn closed 6 years ago
Here's a comparison with logging turned on between using and not using requests.Session()
from justwatch import JustWatch
just_watch = JustWatch(country='GB', use_sessions=False)
Gives for a couple of calls
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
Changing to
from justwatch import JustWatch
just_watch = JustWatch(country='GB', use_sessions=True)
Gives only one connection up front for the same calls
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.justwatch.co
m
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://api.justwatch.com:443 "POST /titles/en_GB/p
opular HTTP/1.1" 200 None
In terms of metrics searching for 20 cases was taking around 4.2 seconds with use_sessions=False
and 3.2 seconds with use_sessions=True
This is a bit of an issue, which I've only picked up on following having my ip blocked when hitting a different website :sob: Easy enough to fix - just have to change requests to requests.Session(). Will post sample logs once I get a chance.