check legislation scrapers - Intros in December on Legistar, but not Councilmatic

[davidmooreppf]

So, looking into why the Freelancers Bill, intro 1017, intro'd last week isn't found on Councilmatic, either by number "1017" or keyword "freelance workers".

I don't know if we pull from this page as a primary source, but here's Brad Lander on Legistar, and on Councilmatic - Intro 1017 isn't on former, of course. But it is findable on main NYC Legistar search, for 1017, introduced on the 7th.

Looking at the minutes from that stated meeting, it also included Intro 128 - couldn't find that one on Legistar - also Intro 609, that's on Legistar and also on Councilmatic.

So, something about 1017's actions? I can't permalink to the results but if you do Legistar Advanced Search for on-agenda on 12/7, it's there in the list of 23 items. Partial screenshot of Legistar search results. But Intro 1014, to pick one at random, is in Legistar but not Councilmatic.

Yesterday was another full council stated meeting - 61 items on Legistar, including intro 1040, which is on Legistar but not on Councilmatic.

Happy to research more, let me know how to help. I can check

[fgregg]

Hmm... okay. It's getting hung on on this vote event

http://legistar.council.nyc.gov/HistoryDetail.aspx?ID=11174708&GUID=09E172AB-B981-4BF1-8247-01846EFCB68A

Particularly on the vote option 'paternity'

Any idea what's going on there?

On Thu, Dec 17, 2015 at 5:45 PM, David Moore notifications@github.com wrote:

Assigned #134 https://github.com/datamade/nyc-councilmatic/issues/134 to @fgregg https://github.com/fgregg.

— Reply to this email directly or view it on GitHub https://github.com/datamade/nyc-councilmatic/issues/134#event-495308786.

773.888.2718

[davidmooreppf]

Will find out tmw from contacts in Council. He does Participatory Budgeting, but I don't know him or his staff closely yet... I mean, if "paternity leave" of some sort affected the scraping, that's remarkably cool.

[fgregg]

Oooh! That would make sense.

On Thu, Dec 17, 2015 at 8:50 PM, David Moore notifications@github.com wrote:

Will find out tmw from contacts in Council. He does Participatory Budgeting, but I don't know him or his staff closely yet... I mean, if "paternity leave" of some sort affected the scraping, that's remarkably cool.

— Reply to this email directly or view it on GitHub https://github.com/datamade/nyc-councilmatic/issues/134#issuecomment-165654850 .

773.888.2718

[davidmooreppf]

I was initially thinking the more highly technical parent-child dynamic code models, of course. High high level stuff brought way down and then way back up.

[davidmooreppf]

Some initial contacts didn't know the answer, but they're looking into it - seems likely it was data entry error. Anyway, what kind of answer is needed? How can we bring in Intro 1017?

[fgregg]

Hmm.. it's in the API http://ocd.datamade.us/bills/?from_organization=ocd-organization/0f63aae8-16fd-4d3c-b525-00747a482cf9&identifier=Int%201017-2015

@cathydeng this will probably get loaded tomorrow?

On Fri, Dec 18, 2015 at 4:08 PM, David Moore notifications@github.com wrote:

Some initial contacts didn't know the answer, but they're looking into it

• seems likely it was data entry error. Anyway, what kind of answer is needed? How can we bring in Intro 1017?

— Reply to this email directly or view it on GitHub https://github.com/datamade/nyc-councilmatic/issues/134#issuecomment-165908344 .

773.888.2718

[cathydeng]

yeah the task runs in the morning. i can go in manually and load it now, since the loaddata task no longer needs to delete existing data

[cathydeng]

kk 1017 is on the site now: http://nyc.councilmatic.org/legislation/int-1017-2015/