search

codeforamerica / councilmatic

A subscription service for city council legislative information, started in Philadelphia.
http://councilmatic.org
58 stars 22 forks source link

Add a note about where the data comes from #15

Closed mjumbewu closed 12 years ago

mjumbewu commented 12 years ago
  1. Just so that people know
  2. So that people know, when there's junk data, that it's not really our fault.
mjumbewu commented 12 years ago

The following was added to the About page:

The data on Philly.Councilmatic is scraped once a day (early in the morning) from the Philadelphia City Council's legislation site. On that site, the data is served through an application called InSite. Many municipal legislative bodies use some version of this software. It is possible that the scraper is able to be used for other cities' InSite instances as well.

Since the data comes from another site, the data quality is only as good as its source. If there are empty communications, that is an issue with the source. Also, Councilmatic may not have up-to-the-minute information about the legislation you are viewing, since it scrapes once per day. Each piece of legislation provides a link to its source for reference.

Councilmatic does add some metadata to the source data: The complete list of words that occur in the source title and attachents, as well as which other legislation mentions or is mentioned by a particular piece of legislation.