BIDS-projects / scraper

Collects data from websites of data science institutions
2 stars 0 forks source link

edited pdf spider and updated items. #21

Closed ExandTran closed 8 years ago

don-han commented 8 years ago

@ExandTran @chewisinho Hey guys, I need you guys to use self.logger.info like shown here for several reasons:

  1. In production, we might want to run it without stdout for better performance and with print, it's difficult to turn off
  2. Similarly, when we want to examine logs, print isn't going to show up on the log file, so we wouldn't really know what your spider is outputting
  3. Since scrapy runs concurrently, we wouldn't know from which spider print is coming from.

You don't specifically have to use self.logging as long as it's a logging system from scrapy. For more information, refer here