List of centrally managed programs

[fgregg]

For our current purposes, a 'program' is something that a student can apply for. Some schools have multiple programs, and other schools only have one.

We need a list of all the programs that cps students can apply to through the central office (this excludes charter/contract schools).

[fgregg]

We can get this through parsing the school guide pdfs here: http://www.cpsoae.org/apps/pages/index.jsp?uREC_ID=189321&type=d&termREC_ID=&pREC_ID=383430&hideMenu=1

Would rather not though.

Let's try:

1. tweet
2. foia
3. parsing

@shua123, you want to start with the tweet?

@JMOCHicago, do you know of this data?

[fgregg]

@evz, @vickiniu and I started by trying to parse the program types field in the data portal. https://data.cityofchicago.org/Education/CPS-Schools-2013-2014-Academic-Year/c7jj-qjvh

This looked really promising, but.. not good enough. Our next step is to parse the appendix of the school guide.

[shua123]

Why not just scrape the data from the CPS school profile pages? The Admissions tab would have the programs directly affected by different admissions types. And the Programs and Clubs tab has others.

[fgregg]

URL please?

[shua123]

The school profiles when you use CPS's find a school: http://cps.edu/Schools/Find_a_school/Pages/findaschool.aspx

For example, Amundsen's: http://schoolinfo.cps.edu/schoolprofile/SchoolDetails.aspx?SchoolId=609695

[derekeder]

Another potential source of data is the Fusion Table that drives the CPS School Locator map.

Here's the Fusion Table: https://www.google.com/fusiontables/DataSource?docid=11zINadePIvkysPdhYUULWV4SqdRL4-XLBjkimyE#rows:id=1

They swap out the table from time to time, but it is always defined here: http://cps.edu/ScriptLibrary/schoollocator2013/scripts/map.js

[bunnyblanco]

Hi,

Just wanted to say I have continued to examin the data in the ISBE Report Card file. I just started reading the documentation on csvkit, so I was doing things the hard way until now. I have to catch up on the email as well.

If anyone wants the school descriptions listed in the report, like their ID number and the location information, I already put together a Python script for that. I will push it to the fork of the main repo I have on github. Once I have a clean import, probably tomorrow, I will generate a pull request.

If you need assistance on any part of the data extraction or analysis, or anything that involves a SQL database, contact me. I have used Sqlite for this project so far, but I can write code for Postgres as well.

Thanks, bcp

Bruce C. Paul

On Wed, Jul 30, 2014 at 9:53 AM, Derek Eder notifications@github.com wrote:

Another potential source of data is the Fusion Table that drives the CPS School Locator map http://cps.edu/Schools/Find_a_school/Pages/schoollocator.aspx.

Here's the Fusion Table: https://www.google.com/fusiontables/DataSource?docid=11zINadePIvkysPdhYUULWV4SqdRL4-XLBjkimyE#rows:id=1

They swap out the table from time to time, but it is always defined here: http://cps.edu/ScriptLibrary/schoollocator2013/scripts/map.js

Reply to this email directly or view it on GitHub https://github.com/open-city/school-admissions/issues/3#issuecomment-50625569 .

[bunnyblanco]

Hi All,

Just to complete the thought and add my $0.02, I pushed the code I wrote to adjust the 2012 - 2013 data file into a workable format, along with the code I wrote to import the school description information. My version of Sqlite doesn't like the number of columns, so I would suggest Postgres to import the entire report. This will be my next contribution, some code to format the headers and some CREATE statements for Postgres, that I will probably generate with csvkit's utilities. It's trivial to do with csvkit, so I will leave this last step for a while, unless anyone needs some assistance. I put my code into the nocsvkit branch of the forked repository I placed on Github.

Let me know if there's any other SQL/Python related database stuff to do. For one thing, writing code to actually break out the report data into several tables with a real schema designed for the business logic proposed for the App might make sense...

Regards, bcp

Bruce C. Paul

On Wed, Jul 30, 2014 at 8:05 PM, Bruce Paul bruce.c.paul@gmail.com wrote:

Hi,

Just wanted to say I have continued to examin the data in the ISBE Report Card file. I just started reading the documentation on csvkit, so I was doing things the hard way until now. I have to catch up on the email as well.

If anyone wants the school descriptions listed in the report, like their ID number and the location information, I already put together a Python script for that. I will push it to the fork of the main repo I have on github. Once I have a clean import, probably tomorrow, I will generate a pull request.

If you need assistance on any part of the data extraction or analysis, or anything that involves a SQL database, contact me. I have used Sqlite for this project so far, but I can write code for Postgres as well.

Thanks, bcp

Bruce C. Paul

On Wed, Jul 30, 2014 at 9:53 AM, Derek Eder notifications@github.com wrote:

Another potential source of data is the Fusion Table that drives the CPS School Locator map http://cps.edu/Schools/Find_a_school/Pages/schoollocator.aspx.

Here's the Fusion Table: https://www.google.com/fusiontables/DataSource?docid=11zINadePIvkysPdhYUULWV4SqdRL4-XLBjkimyE#rows:id=1

They swap out the table from time to time, but it is always defined here: http://cps.edu/ScriptLibrary/schoollocator2013/scripts/map.js

Reply to this email directly or view it on GitHub https://github.com/open-city/school-admissions/issues/3#issuecomment-50625569 .

[fgregg]

@derekeder it looks like that data source has the same stuff as the data portal

[fgregg]

@bunnyblanco awesome!! I'll take a look early next week!

[achtor]

Hey, I'm at hack night but can't seem to locate the education group. Where are you guys?

On Thu, Jul 31, 2014 at 8:25 PM, Forest Gregg notifications@github.com wrote:

@bunnyblanco https://github.com/bunnyblanco awesome!! I'll take a look early next week!

— Reply to this email directly or view it on GitHub https://github.com/open-city/school-admissions/issues/3#issuecomment-50839677 .

[elnaz]

Sorry Peter, I hope you were able to find some people. Josh and I were on that field trip to the Hideout ed talk that I had emailed about. Next week both Josh and I are unable to attend hack night, but we'll back on Aug 19.

Elnaz

On Tue, Aug 5, 2014 at 7:54 PM, Peter Xu notifications@github.com wrote:

Hey, I'm at hack night but can't seem to locate the education group. Where are you guys?

On Thu, Jul 31, 2014 at 8:25 PM, Forest Gregg notifications@github.com wrote:

@bunnyblanco https://github.com/bunnyblanco awesome!! I'll take a look early next week!

— Reply to this email directly or view it on GitHub < https://github.com/open-city/school-admissions/issues/3#issuecomment-50839677>

.

— Reply to this email directly or view it on GitHub https://github.com/open-city/school-admissions/issues/3#issuecomment-51281551 .

[derekeder]

I am scraping the list of schools by program type off this PDF: http://www.cpsoae.org/Glossary_%20Appendix_%20and%20Index%20--%20English.pdf

Doc started: https://docs.google.com/spreadsheets/d/1Zi4Ot-yLGOOWw5wfj5iIelLoIIaTaHbHjM1A6D3hUbU/edit#gid=0

[derekeder]

Done! https://docs.google.com/spreadsheets/d/1Zi4Ot-yLGOOWw5wfj5iIelLoIIaTaHbHjM1A6D3hUbU/edit#gid=0