search

openaustralia / australian_local_councillors_images

Stores a copy of Australian local councillor images from Popolo on S3. Runs on morph.io
https://morph.io/openaustralia/australian_local_councillors_images
0 stars 0 forks source link

Work out how to load a new council's images quickly #4

Closed henare closed 8 years ago

henare commented 8 years ago

In the PlanningAlerts readme we breezily say to run this on morph.io to load images for a council you're adding. However this takes an hour right now so that's no good!

henare commented 8 years ago

I think the easiest way to solve this is to run it locally. We can probably add arguments to the scraper so it only tries to load the one council you specify.

equivalentideas commented 8 years ago

I think the easiest way to solve this is to run it locally.

@henare Is it much faster locally than on morph.io? Why’s that?

henare commented 8 years ago

I've not run it all the way through locally so I don't know.

I meant it's easier fit into your workflow locally as you can futz with the settings easily without having to fiddle with the scraper on morph.

equivalentideas commented 8 years ago

For the record, it is way faster to run this locally for some reason.

Also, the ability to target just a single authority was added in https://github.com/openaustralia/australian_local_councillors_images/commit/c35bbe7b3c4df3dc7ebef7c3b192c521b79baf5e

This, and the ability to target just one state (https://github.com/openaustralia/australian_local_councillors_images/commit/ecdd68706fa552aff64a186f553dd10dcb0b0f96 ) make this much quicker.

I'll add that tip to the docs for administrators in PlanningAlerts.

henare commented 8 years ago

Cool, I think we can say this is fixed then.