search

cenpy-devs / cenpy

Explore and download data from Census APIs
Other
183 stars 44 forks source link

Add requests.session to REST classes #141

Closed ronnie-llamado closed 11 months ago

ronnie-llamado commented 3 years ago

Add a requests.session to APIConnection, TigerConnection and ESRILayer. Creates persistent HTTP connection to be reused when requesting from the same domain. Currently each GET request opens a new connections, transacts and then closes the connection.

Closes #140

ronnie-llamado commented 3 years ago

The following code was used as a benchmark.

chicago = \
    cenpy.products.ACS(2017).from_place("Chicago, IL", level="tract", variables=["B00002*", "B01002H_001E"])

There were two noticeable changes:

  1. There were 9 less connections opened/closed.
  2. Using the timeit module, the refactor shaved ~3s on average from executing the code above. 
original      (seconds) : [24.64, 27.20, 25.48, 23.19, 23.36, 24.66, 29.26, 25.37, 26.27, 22.84] : 25.23 (avg) 
with refactor (seconds) : [26.96, 19.80, 21.40, 23.00, 22.00, 19.68, 23.19, 20.46, 19.91, 25.15] : 22.16 (avg)