Refactored the Career OneStop scraper, the HUD scraper and the California Food Resources scraper to use a parent-child class inheritance structure, an idea that Sanjoy from the Hackathon put forth and got started.
The gist is that we have a Python class that represents a base scraper and therefore contains parameters and instance methods (functions) that apply to all the scrapers. Each individual scraper is then an instance of a child class which inherits from this base scraper and adds any tweaks that are idiosyncratic to that data source.
I think this should make the process of writing scrapers for the remaining data sources a MUCH smoother process, because we won't be starting from scratch every time.
Refactored the Career OneStop scraper, the HUD scraper and the California Food Resources scraper to use a parent-child class inheritance structure, an idea that Sanjoy from the Hackathon put forth and got started.
The gist is that we have a Python class that represents a base scraper and therefore contains parameters and instance methods (functions) that apply to all the scrapers. Each individual scraper is then an instance of a child class which inherits from this base scraper and adds any tweaks that are idiosyncratic to that data source.
I think this should make the process of writing scrapers for the remaining data sources a MUCH smoother process, because we won't be starting from scratch every time.