cobalt-uoft / uoft-scrapers

Public web scraping scripts for the University of Toronto.
https://pypi.python.org/pypi/uoftscrapers
MIT License
48 stars 14 forks source link

Scrape useful layers from map.utoronto.ca #20

Closed qasim closed 8 years ago

qasim commented 8 years ago

Check out http://map.utoronto.ca/ and how there are so many layers. If any of the layers make sense, let's build a scraper for them.

kashav commented 8 years ago

Started a LayersScraper subclass for all layers' scrapers. Don't know if this is even required, but it seems like scraping each layer is gonna be done similarly.

Also, does it make sense to add this alongside the Scraper superclass? (not really sure if this is the Pythonic way of doing things)

qasim commented 8 years ago

It's a good idea; lots of duplicate code otherwise.

In terms of where to place it, not too sure either. Since its a very basic superclass, we can move scrapers/layers/ to scrapers/scraper/layers/. That way the root directories in scrapers/ will have the scraper/ where all the superclasses are, and the rest of the folders are actually the real scrapers.

qasim commented 8 years ago

Some other layers we should consider:

qasim commented 8 years ago

Everything I'd like to see for now from layers has its own issue. Closing this, but will reference it when we decide to go for more.