Greece: broken scraper (out of memory)

everypolitician / everypolitician-data

data for national legislatures worldwide

http://everypolitician.org/

235 stars 54 forks source link

Open tmtmtmtm opened 7 years ago

tmtmtmtm commented 7 years ago

https://morph.io/tmtmtmtm/hellenic-parliament is failing with a morph 255 error (presumably out of memory)

ondenman commented 7 years ago

It seems to be running fine locally (without scraped page archiving turned on).

ondenman commented 7 years ago

The scraper has been broken for two months. The last commit changed the unique index key. I've opened a PR to DROP the table on each run.

The commit before the index change turned on archiving.

ondenman commented 7 years ago

It will need to be refactored for Scraped first to pass rubocop checks.