rs-fraser
commented
2 months ago @mKowalski256 @rs-nicof I am off this coming week. I've tested out the changes I made in response to the inefficiency @rs-nicof pointed out yesterday on pangolin and all seems to be working fine. Please feel free to make any changes and merge this PR when you guys are happy with it.
This PR improves the performance of:
Previously in both of the above features, we would first limit the list of records which could be searched in various ways, including by permissions and filtering.
This caused slow performance in cases such as on pangolin85 as sysadmin or for large customer installs, where there are lots of documents.
On the UI search which performs well, we first search all of the indexed documents, then apply any filters or permission-based view rights to the smaller list of search results.
In addition, in the case of an empty search or 'View All' on the workspace where we list all documents, we would first of all loop through each of the users which the subject user can view (e.g. members of a PI subjects groups, or everyone for a sysadmin) and perform a db call for each to get their documents. Instead, the db call now takes a list of user ids and performs an IN query so that there only needs to be 1 db call.
The above changes have improved performance on pangolin85, operating as sysadmin, as follows:
There is still probably further we could go with this as in the case of an empty (i.e. all records) on an instance with 100k records in scope this will still be slow, but hopefully won't time out. I think another avenue to address would be the way we handle permissions checking.
Additionally, there are a bunch of code cleanup changes included in the files which I touched.