expand exception handler

KaylaCrush / advocacy-maps

The good governance project (GGP) is a non-partisan democracy reform group.

https://mapletestimony.org

MIT License

0 stars 1 forks source link

expand exception handler #32

Closed KaylaCrush closed 1 year ago

KaylaCrush commented 1 year ago

Ultimately for this project to be stable I'm going to need to have decent exception handling. When the program encounters a table it can't process, it should save the url or html to a file, raise a warning of some kind, and keep going.

[x] Create a try/catch around the most error-prone areas of the code (creating the pandas df) so we can keep going when it breaks
[ ] Save the url or html to disk so someone can go debug it

KaylaCrush commented 1 year ago

currently just printing a message to the terminal with when this happens

KaylaCrush commented 1 year ago

this might require me to integrate the helper functions like create_table into the main body of the lobbyingdatapage class so that I have access to more info. Right now the dataframe_exception method only has access to the divided_text and columns that caused the error

KaylaCrush commented 1 year ago

so, this sounds insanely cocky, but so far my new beautiful soup refactor hasn't had any of the kinds of errors that were cropping up in the past. Knock on wood.