Raw Spider - Githubissues

BIDS-projects / scraper

Collects data from websites of data science institutions

2 stars 0 forks source link

Closed alvinwan closed 8 years ago

alvinwan commented 8 years ago

simple, raw HTML output - follows dlab spider setup
makefile with simple deploy target make deploy path=[path/to/spider] and simple crawl target make crawl project=[project] spider=[spider]

alvinwan commented 8 years ago

Yeah, I copied the CSV into the code as a last resort. ScrapingHub doesn't copy non-Python files onto server during deployment.

Didn't know about the default parse. My bad; I thought it was convention to name the method parse.

don-han commented 8 years ago

Ah, that's interesting about ScrapingHub.