Generalized Scraper Framework

[BCarlson1512]

Long term infrastructure goal:

• Describe the scraping process on a high level
• Generate a list of states to follow
• Create a generalized and reusable framework for all future scrapers to follow
• Pass into an endpoint/docker image that can be run on a timely basis

[BCarlson1512]

This is primarily for when we want to expand to other schools and widen our overall scope of the project.

[BCarlson1512]

Addressing our first point

Important states may include

FIND_COURSE_CALENDAR FIND_COURSES FIND_COURSE_DETAILS PARSE_COURSES

FIND_PROGRAMS_CALENDAR FIND_PROGRAMS FIND_PROGRAM_DETAILS PARSE_PROGRAMS