cobalt-uoft / uoft-scrapers

Public web scraping scripts for the University of Toronto.
https://pypi.python.org/pypi/uoftscrapers
MIT License
48 stars 14 forks source link

New format + README organization #46

Closed qasim closed 8 years ago

qasim commented 8 years ago

This is the PR that will reflect new format proposals from #39, along with a newer proposal for scraper organization for ones that require a different scraper for each campus.

Let's take Exams for an example. If a user calls Exams.scrape(), it should subsequently perform the following:

Of course, you can directly call the above APIs still. This just adds a convenient alias to scrape all of them with one call. All of these scrapers remain in the global namespace, so no changes with that.

Note: Only the README has changed so far; I will be reflecting changes to the actual codebase as well once all of the scrapers currently in development / almost complete are merged.

qasim commented 8 years ago

On another note: the second proposal will also allow for more abstraction within scrapers. The Exams scraper can now hold methods that are common to the sub-scrapers.

qasim commented 8 years ago

Some other notable changes:

qasim commented 8 years ago

Good to go from my end.