search

18F / 2015-foia-hub

A consolidated FOIA request hub.
Other
49 stars 17 forks source link

Add Request Processing Time to Data #179

Closed rjmajma closed 9 years ago

rjmajma commented 9 years ago

Conducting an advanced search on FOIA.gov allows you to query and download the processing time for FY13 FOIA requests. There are around 100 CSVs that need to be downloaded. One for the average processing time for simple and complex requests for each of the 99 agencies as a whole and then downloading each individual agency, which then breaks down to the component level.

Displaying the average processing time for "simple" and "complex" requests will help users better anticipate how long their FOIA request may take to be processed.

Work to consider for completion (this will warrant breaking out into distinct tickets):

jackiekazil commented 9 years ago

@rjmajma in our sprint planning, we have the following. I am going through this now. You will see that I have added some info to this ticket to make sure we have actual dev tickets connected for each component of the work. This is what we have in the sprint doc... 

Get average processing times from foia.gov for agencies, components. (Jackie)
Store average processing times. (Gabriel)
Display average processing times. (Gabriel)

@rjmajma -- can you clarify what is "Provide this through the API" vs "Display average processing times"?

@ramirezg -- can you attach ticket numbers for the ticket numbers to this ticket for the tickets that you have created?

rjmajma commented 9 years ago

"Displaying the average processing times..." simply speaks to the user need this task addresses.

I'm not sure what isn't clear about our need to provide this through the API. This information should be reflected there, since we are using the API as the foundation for what goes on the agency landing pages (where this will be displayed).

rjmajma commented 9 years ago

@vz3 and @jtag, this information will appear somewhere on the template for you to play around with later. (Template work: #186)

geramirez commented 9 years ago

After looking over the data @jackiekazil and I decided to include all of them into to a new model with a foreign key to the agency and office. One or two fields don't really covey the time frame these requests take and adding everything to the agency and office models would make everything too messy.

Below is a list of the attributes included in the data. They also range from 2008-2013. 

Complex-Average No. of Days
Complex-Highest No. of Days
Complex-Lowest No. of Days
Complex-Median No. of Days
Expedited Processing-Average No. of Days
Expedited Processing-Highest No. of Days
Expedited Processing-Lowest No. of Days
Expedited Processing-Median No. of Days
Simple-Average No. of Days
Simple-Highest No. of Days
Simple-Lowest No. of Days
Simple-Median No. of Days

Validation: The data contained a mix of "0"s and NAs to denote a missing value. All of them were normalized to "NA".

Thoughts, @khandelwal

Ref PR#56

jtag commented 9 years ago

Oh I see (cc @vz3 ) there's: Simple, Complex, and Expedited categories, not just Simple and Complex.

Just chatting with Chris Papasian here in SF and he suggested out of these, median is the best. He said ideally if we could get more than the median and be able to express that "80-90% of requests take N days" that would be more accurate. cc @rjmajma

rjmajma commented 9 years ago

Expedited is there, but so, so few are granted. Less than 2,000 of the over 700,000 requests were processed as expedited requests. Would this data point be useful to show folks making a request?

We decided on going with the median number, even though we may call it something else in the copy.

For later iterations, it would be worth digging into the Chief FOIA Officer reports to see if we can get that level of granularity. I'll create a new ticket and add it to the backlog.

rjmajma commented 9 years ago

This seems to be done. Thanks for all your hard work, everyone!