search

openelections / fec_results_generator

JSON API Generator for FecResults Library
MIT License
4 stars 1 forks source link

Congressional Results #3

Open KirkHadley opened 10 years ago

KirkHadley commented 10 years ago

Congressional Results curled from the API (i.e. not in Ruby) aren't formatted as proper JSON; instead each record is a string d/t quotes at each record. i.e. [" <-- this quote {\"year\":2012,\"date\":null,\"chamber\":\"H\",\"state\":\"AL\",\"district\":\"01\",\"fec_id\":\"H2AL01077\",\"incumbent\":true,\"candidate_last\":\"Bonner\",\"candidate_first\":\"Jo\",\"candidate_suffix\":null,\"candidate_name\":\"Bonner, Jo\",\"party\":\"R\",\"primary_votes\":48702,\"primary_pct\":55.54959907839358,\"primary_unopposed\":false,\"runoff_votes\":null,\"runoff_pct\":null,\"general_votes\":196374,\"general_pct\":97.85624588889553,\"general_unopposed\":false,\"general_runoff_votes\":null,\"general_runoff_pct\":null,\"general_combined_party_votes\":null,\"general_combined_party_pct\":null,\"general_winner\":true,\"notes\":null}" <-- and this quote

dwillis commented 10 years ago

@KirkHadley, what kind of tool are you using on the command line? I've tried some of the API responses using jq and jsontool and it seems to handle the JSON just fine.

KirkHadley commented 10 years ago

I curled from the command line and jq recognized it as JSON, but instead of each election being a standard json record, they were read in as strings. In an unsuccessful attempt to avoid brewing up any awk/grep/sed incantations, I thought it might have just been a weird jq bug so I just read the files into python. Unfortunately I got the same result there as well. Then I tried using pycurl, thinking that maybe there was something weird about my the request maybe, but still got the same result [i.e. I had a list of objects (the elections), but each object was a string where it should have been a dict]. I took a look at the raw json file at the endpoint and I think I found the problem (Note: the following is the first record at this endpoint: http://openelections.github.io/fec_results/api/2012/congress/results.json):

[_" {\"year\":2012,\"date\":null,\"chamber\":\"H\",\"state\":\"AL\",\"district\":\"01\",\"fec_id\":\"H2AL01077\",\"incumbent\":true,\"candidate_last\":\"Bonner\",\"candidate_first\":\"Jo\",\"candidate_suffix\":null,\"candidate_name\":\"Bonner, Jo\",\"party\":\"R\",\"primary_votes\":48702,\"primary_pct\":55.54959907839358,\"primary_unopposed\":false,\"runoff_votes\":null,\"runoff_pct\":null,\"general_votes\":196374,\"general_pct\":97.85624588889553,\"general_unopposed\":false,\"general_runoff_votes\":null,\"general_runoff_pct\":null,\"general_combined_party_votes\":null,\"general_combined_party_pct\":null,\"general_winner\":true,\"notes\":null} "_,

The issue was that the record (in fact every record) was wrapped in quotes (bolded). However, I wonder if this is only a problem because I was using Python and not ruby? I know that backslashes make python angry sometimes. Also, there were some strangely formatted candidate names. Like this candidate from AL in 2012: "candidate_first\":\"Penny \"Colonel\"\". Python really didn't like the quotes in the middle of a dict entry.

Weird right?

On Sun, May 11, 2014 at 4:30 PM, Derek Willis notifications@github.comwrote:

@KirkHadley https://github.com/KirkHadley, what kind of tool are you using on the command line? I've tried some of the API responses using jq and jsontool and it seems to handle the JSON just fine.

— Reply to this email directly or view it on GitHubhttps://github.com/openelections/fec_results_generator/issues/3#issuecomment-42782180 .