2022-06-18: Minimize chance to unsubscribe because courts are slow to update new court dates

[ejaxon]

It can happen that there is a delay between a scheduled court date that is either continue or at which the defendant fails to appear and a new court date showing up in the state courts website. When that happens, we can end up purging a subscriber prematurely.

We have changed the language of the unsubscribe notification to be less categorical and delayed the actual purge for 7 days, but the update function can still end up causing a defendant to be purged. This is because we delete their past court dates and then replace them with the new ones. If there are no new ones, the defendants and subscriptions will end up being purged anyway.

Because we have multiple update functions interacting with sometimes unreliable external information sources, this needs to be carefully thought through. I'd also like to diagram it so that we can share with court officials and County communications folks.

[ejaxon]

Update on the plan for this. Right now we purge expired court dates after a specified number of days. However, we also update records for every defendant once a week, which also erases any expired court dates. So the purge is basically useless extra work, and makes fixing this more complicated. So the fix here is as follows:

• [x] Kill the purge of cases from purge-and-update-subscriptions.js (this can and should be done independently of the rest since it basically just simplifies the code).

• [x] Add a _last_valid_casesdate column to the defendants table and only update when there are non-zero case matches (see lines 28-33 in update_defendants.js).

• [x] Modify purge-and-update-subscriptions.js to delete defendents not when they have no cases, but only when _last_valid_casesdate is more than a specified number of days old (maybe 30 days just to be safe, but in any case should allow at least 2 updates, so more than 2 weeks).

• [x] When pushing to production, make sure to carefully run the migration, then set all the _last_valid_casesdate values to the current day. Also check the values of things in cn_configuration.