search

CountCulture / twfy_local_parser

Local scraper parser to scrape and parse local authority data
http://theyworkforyoulocal.com
20 stars 1 forks source link

API call for all members #5

Closed adrianshort closed 14 years ago

adrianshort commented 14 years ago

Hi,

Please could you enable an API call that returns all the members, eg:

/members.xml

This would be very useful in combination with the YQL open data table bindings which already works on the individual member pages, eg:

/members/187.xml

Thanks.

Adrian

CountCulture commented 14 years ago

Presumably this would be OK if it's paginated. Otherwise we're talking about a huge response. Wouldn't want to think what the server implications would be otherwise

adrianshort commented 14 years ago

Pagination would be fine.

However, the purpose of the exercise, for me, would be to enable some kind of search facility on the members which would generally limit the number of results returned. For those that wanted to return all the members I'm sure they wouldn't mind paging through the results too.

For example, it might be handy to extract (using YQL if necessary) the names, council names and ward names of all Labour councillors at London councils. Doing something like that would imply having a search parameter for each column in that table.

YQL caches all calls to your API but there's no guarantee that anyone else would.

adrianshort commented 14 years ago

Additionally, you could generate a zipped XML dump of the whole database once a day.

CountCulture commented 14 years ago

Searches on various attributes is something I've been meaning to do, so prob that's the best solution (and then combining with with pagination, maybe with allowing up to 200 per page). Have got a couple of urgent things I've got to finish of over the next week, then will have a go at this.

CountCulture commented 14 years ago

Adrian Apologies for the delay. Implemented this a while ago, but forget to update this ticket. You can now paginate through all members at http://openlylocal.com/members

CountCulture commented 14 years ago

Having said that I haven't yet opened up API access to the members list accessed in this way. Will get onto it next week.

CountCulture commented 14 years ago

Members list now available through api