search

ContentMine / quickscrape

A scraping command line tool for the modern web
MIT License
259 stars 43 forks source link

"TypeError: Cannot read property 'actions' of null" if wrong scraper used #48

Closed petermr closed 9 years ago

petermr commented 9 years ago

If the wrong scraper is used quickscrape should fail gracefully:

localhost:elsevier pm286$ quickscrape -u http://www.sciencedirect.com/science/article/pii/S0031942215000965 -s ../../journal-scrapers/scrapers/nature.json 
info: quickscrape launched with...
info: - URL: http://www.sciencedirect.com/science/article/pii/S0031942215000965
info: - Scraper: ../../journal-scrapers/scrapers/nature.json
info: - Rate limit: 3 per minute
info: - Log level: info
info: urls to scrape: 1
info: processing URL: http://www.sciencedirect.com/science/article/pii/S0031942215000965

TypeError: Cannot read property 'actions' of null
    at /Users/pm286/.nvm/v0.10.38/lib/node_modules/quickscrape/node_modules/thresher/lib/thresher.js:104:16
    at Request._callback (/Users/pm286/.nvm/v0.10.38/lib/node_modules/quickscrape/node_modules/thresher/lib/url.js:60:5)
    at Request.self.callback (/Users/pm286/.nvm/v0.10.38/lib/node_modules/quickscrape/node_modules/thresher/node_modules/request/request.js:368:22)
    at Request.emit (events.js:98:17)
    at Request.<anonymous> (/Users/pm286/.nvm/v0.10.38/lib/node_modules/quickscrape/node_modules/thresher/node_modules/request/request.js:1219:14)
    at Request.emit (events.js:117:20)
    at IncomingMessage.<anonymous> (/Users/pm286/.nvm/v0.10.38/lib/node_modules/quickscrape/node_modules/thresher/node_modules/request/request.js:1167:12)
    at IncomingMessage.emit (events.js:117:20)
    at _stream_readable.js:944:16
    at process._tickCallback (node.js:448:13)
localhost:elsevier pm286$