sjDan2003 / Cookbook

0 stars 0 forks source link

Properly handle all urllib urlopen exceptions #11

Open sjDan2003 opened 5 years ago

sjDan2003 commented 5 years ago

The website scrapper currently doesn't handle many exceptions that can occur if there is an issue connecting to a website. Investigate how unit testing's patch system can mimic an exception, document how this can be done in this case, and create unit tests that will test for these exceptions.

sjDan2003 commented 5 years ago

I determined that using normal patches and mocks was not working because I wrapped the urlopen function in a context manager. The context manager itself needed to be mocked as well. I found the information on the following sites to be helpful: https://stackoverflow.com/questions/28850070/python-mocking-a-context-manager https://chase-seibert.github.io/blog/2015/06/25/python-mocking-cookbook.html# https://jsatt.com/blog/mocking-context-managers-in-python/

sjDan2003 commented 5 years ago

Detailed information regarding each HTTPError code can be found here: https://tools.ietf.org/html/rfc2616.html https://docs.python.org/3/library/http.html I think the best way to handle most 4xx and 5xx codes is to provide the user as much information as possible via pop up, and prevent the recipe from being imported.