search

18F / open-data-maker

make it easy to turn a lot of potentially large csv files into easily accessible open data
Other
200 stars 135 forks source link

Searching by 2013.completion.title_iv.low_inc.completed_by.4yrs gives Internal Server Error #230

Open hollyallen opened 8 years ago

hollyallen commented 8 years ago

Searching by 2013.completion.title_iv.low_inc.completed_by.4yrs gives Internal Server Error

Example: https://api.data.gov/ed/collegescorecard/v1/schools?fields=2013.completion.title_iv.low_inc.completed_by.4yrs&api_key=

ultrasaurus commented 8 years ago

here's what I see in the log when I run this query on the dev server:

2015-11-18T02:34:18.52-0800 [RTR/0]      OUT ccapi-dev.18f.gov - [18/11/2015:10:34:18 +0000] "GET /v1/schools?sort=2011.earnings.6_yrs_after_entry.percent_greater_than_25000%3Adesc&school.name=arizona&school.degrees_awarded.predominant__range=2..3&school.operating=1&fields=id%2Cschool.name%2Cschool.city%2Cschool.state%2C2013.student.size%2Cschool.ownership%2Cschool.degrees_awarded.predominant%2C2013.cost.avg_net_price.overall%2C2013.completion.rate_suppressed.overall%2C2011.earnings.10_yrs_after_entry.median%2C2011.earnings.6_yrs_after_entry.percent_greater_than_25000%2Cschool.under_investigation HTTP/1.1" 200 0 5096 "-" "Pingdom.com_bot_version_1.4_(http://www.pingdom.com/)" 127.0.0.1:16546 x_forwarded_for:"67.228.213.178" vcap_request_id:fd7d0316-7f25-4bc0-465f-ae52fcbf0d59 response_time:0.024106778 app_id:90907452-4731-4a93-a511-132eef6b1554
2015-11-18T02:34:18.52-0800 [App/0]      OUT I, [2015-11-18T10:34:18.527964 #34]  INFO -- : ES query time (ms): 14 ; Query fetch time (s): 0.017751216888427734 ; result: {"took"=>14, "timed_out"=>false, "_shards"=>{"total"=>1, "successful"=>1, "failed"=>0}, "hits"=>{"total"=>13, "max_score"=>nil, "hits"=>[{"_index"=>"production-school-data", "_type"=>"document", "_id"=>"454245", "_score"=>nil, "fields"=>{"id"=>[454245], "school.city"=>["Phoenix"], "school.under_investigation"=>[0], "school.state"=>["AZ"], "school.degrees_awarded.predominant"=>[3], "2013.cost.avg_net_price.overall"=>[19540], "2011.earnings.6_yrs_after_entry.percent_greater_than_25000"=>[0.91786450
2015-11-18T02:34:29.31-0800 [App/0]      OUT D, [2015-11-18T10:34:29.313496 #46] DEBUG -- : -----> APP GET {}
2015-11-18T02:34:29.31-0800 [App/0]      OUT I, [2015-11-18T10:34:29.313675 #46]  INFO -- : search terms:{}
2015-11-18T02:34:29.31-0800 [App/0]      OUT I, [2015-11-18T10:34:29.313742 #46]  INFO -- : FULL_QUERY: {:index=>"production-school-data", :type=>"document", :body=>{:from=>0, :size=>20, :query=>{:match_all=>{}}, :fields=>["2013.completion.title_iv.low_inc.completed_by.4yrs"], :_source=>false}}
2015-11-18T02:34:29.45-0800 [App/0]      OUT I, [2015-11-18T10:34:29.457969 #46]  INFO -- : ES query time (ms): 141 ; Query fetch time (s): 0.14406323432922363 ; result: {"took"=>141, "timed_out"=>false, "_shards"=>{"total"=>1, "successful"=>1, "failed"=>0}, "hits"=>{"total"=>7804, "max_score"=>1.0, "hits"=>[{"_index"=>"production-school-data", "_type"=>"document", "_id"=>"166498", "_score"=>1.0}, {"_index"=>"production-school-data", "_type"=>"document", "_id"=>"169071", "_score"=>1.0}, {"_index"=>"production-school-data", "_type"=>"document", "_id"=>"190752", "_score"=>1.0}, {"_index"=>"production-school-data", "_type"=>"document", "_id"=>"208187", "_score"=>1.0
2015-11-18T02:34:29.45-0800 [App/0]      ERR 2015-11-18 10:34:29 - NoMethodError - undefined method `keys' for nil:NilClass:
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/lib/data_magic.rb:98:in `block in search'
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/lib/data_magic.rb:93:in `map'
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/lib/data_magic.rb:93:in `search'
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/app/controllers.rb:53:in `block (2 levels) in <top (required)>'
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/vendor/bundle/ruby/2.2.0/gems/padrino-core-0.12.5/lib/padrino-core/application/routing.rb:504:in `call'
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/vendor/bundle/ruby/2.2.0/gems/padrino-core-0.12.5/lib/padrino-core/application/routing.rb:504:in `block in route'
2015-11-18T02:34:29.45-0800 [App/0]      ERR    /home/vcap/app/vendor/bundle/ruby/2.2.0/gems/padrino-core-0.12.5/lib/padrino-core/ext/http_router.rb:33:in `[]'
ultrasaurus commented 8 years ago

this may be fixed with: https://github.com/18F/open-data-maker/pull/228

ultrasaurus commented 8 years ago

confirmed with local dev env that this has been resolved

ultrasaurus commented 8 years ago

@hollyallen should we create some kind of state for fixed and not deployed? for now will label as "accept" (which I think means it is ready for someone to accept it, which would technically only be true once it has been deployed to staging)

hollyallen commented 8 years ago

Agreed that technically the Accept label shouldn't be applied until it is in staging, but for now we can leave that label on.