search

datamade / nyc-council-councilmatic

NYC Council version of Councilmatic
MIT License
8 stars 3 forks source link

Are we purposefully not showing sponsor information? #64

Closed fgregg closed 6 years ago

fgregg commented 6 years ago

https://nyc-council-staging.datamade.us/legislation/int-1786-2017/

reginafcompton commented 6 years ago

The logic for bill sponsorships follows:

Data scraped from Legistar API....NYC scraper Data imported Councilmatic Data rendered in templates

It looks like we did not scrape the sponsors for this particular bill. The matter has a LastModifiedUtc of 2017-12-11T17:46:18.09, and the sponsor has a LastModifiedUtc of 2017-12-06T18:32:27.85. It seems like we could have missed this bill, given the irregularity of our scrapes these last days. @hancush - does that seem reasonsible?

N.B. Many bills do have sponsorships: https://nyc-council-staging.datamade.us/legislation/t-2017-6565/ https://nyc-council-staging.datamade.us/legislation/t-2017-5727/ https://nyc-council-staging.datamade.us/legislation/t-2017-5867/ https://nyc-council-staging.datamade.us/legislation/t-2016-4004/

hancush commented 6 years ago

We've had the cron turned off the past two days, so it's definitely possible we're just behind.

reginafcompton commented 6 years ago

@hancush - the bill in question does not have sponsorships in the OCD API, though I see a sponsor in the Legistar API. Can you look into this on the scraper-side?

hancush commented 6 years ago

This should take care of it. https://github.com/opencivicdata/python-legistar-scraper/pull/63

hancush commented 6 years ago

Unfortunately, I think it also means a big scrape (again).

reginafcompton commented 6 years ago

OCD API and Councilmatic site now have sponsors