search

mysociety / alaveteli

Provide a Freedom of Information request system for your jurisdiction
https://alaveteli.org
Other
386 stars 196 forks source link

Public body csv upload should not list all bodies in database not in spreadsheet when this is a big number #2018

Open crowbot opened 9 years ago

crowbot commented 9 years ago

When people are uploading a small supplementary spreadsheet, the time taken to render this list is greater than the timeout of our sites. When the upload file is a fraction of the db size (less than half), we should just give a note that there are existing bodies that haven't been updated.

crowbot commented 9 years ago

I looked into this a bit more, and it seems like most of the time was spent in N+1 querying on the translations table and internal admin authority record. I've fixed those issues in a21420efff11968c58c9d220ea5f9974142056b6.

RichardTaylor commented 8 years ago

While this is no longer a problem on the grounds of the page timing out I think showing the full body list is still surprising and confusing when adding a small number of extra bodies to site like WhatDoTheyKnow with many thousands of bodies.

The assumption appears to be that there's a CSV file with all the bodies in being maintained separately to the Alaveteli database, which isn't the case with, at least, WhatDoTheyKnow.