cfpb / regulations-site

(DEPRECATED) Web interface for viewing U.S. federal regulations and other regulatory information
Other
28 stars 43 forks source link

Remove use of threads when retrieving layers #846

Closed chosak closed 6 years ago

chosak commented 6 years ago

This change removes the use of multiple threads when retrieving layer JSON. Instead of spawning a new thread for each of the ten layers that need to be retrieved on a request, the layers are retrieved sequentially.

While this may have the effect of slowing down data retrieval somewhat, it simplifies how predictably this code can be expected to run in a web server. This PR is intended for use alongside #845 which removes dependence on an external API entirely, instead making calls via Python directly.

The best way to test this is either with a local setup with regulations-core with a complete production database or by pointing your EREGS_API_BASE to a live endpoint with loaded regs:

$ EREGS_API_BASE=https://www.consumerfinance.gov/eregs-api/ python manage.py runserver

Click around a complex reg, e.g. somewhere in Reg Z and observe how long it takes to load the content. Compare with the master branch to see how much of an impact this change has.