search

mysociety / alaveteli

Provide a Freedom of Information request system for your jurisdiction
https://alaveteli.org
Other
389 stars 195 forks source link

Calling PublicBody#expire_requests causes platform stability issue #7101

Closed gbp closed 1 year ago

gbp commented 2 years ago

Just had a issue where most of our WDTK passenger processes were tied up and weren't able to response to web traffic.

Which root cause is due to calling PublicBody#expire_requests

This happens for at least two (maybe more?) occasions.

  1. PublicBody#url_name is changed
  2. CensorRule applicable to a public body's requests is created/updated/destroyed

In absence of #1240 we should ensure this works without blocking the passenger processes.

Initial attempt at fixing this is at https://github.com/mysociety/alaveteli/pull/6967

RichardTaylor commented 2 years ago

Noting that today's issue related to adding/editing a body-wide censor rule to a body with lots of requests - https://github.com/mysociety/whatdotheyknow-private/issues/20#issuecomment-1154038699

A previous similar feeling issue arose when renaming a set of authorities which had a reasonable number of requests made to them. This was in relation to name changes on Welsh County Councils. See https://github.com/mysociety/whatdotheyknow-theme/issues/1131 and https://github.com/mysociety/whatdotheyknow-theme/issues/1132

Perhaps the fix here will address the issue in both cases?

garethrees commented 2 years ago

Noting that there's a bunch of different implementations for essentially the same thing across different models, so we should think about this from the point of view of coalescing them into a common Reindexable concern (https://github.com/mysociety/alaveteli/issues/6931).

garethrees commented 1 year ago

@gbp is this resolved by https://github.com/mysociety/alaveteli/issues/1240?

gbp commented 1 year ago

Yes, this should all be fixed