Solve the issue of #3250 narrowly, where we noticed it. This is to make sure that even if #3252 doesn't delete 100% of entries and some slip through it has no effect on processing pipeline updates. This also means existing instances don't need to clean up processed data to avoid #3250.
It'd be nice if we could rewrite the single big SQL update statement into chunks so we can do some informative logging of intermediate results to help with debugging updates that are in process, see #3249 - but that's for another time.
Once we have foreign key constraints we can probably remove the inner join again.
Testing
I tested deleting some sequences then updating prepro version and it worked (PR #3256)
fixes #3250 independently of #3252, but that PR is worth it anyways as we don't want processed data lying around for deleted sequence entries.
preview URL: https://ignore-deleted.loculus.org
Summary
Solve the issue of #3250 narrowly, where we noticed it. This is to make sure that even if #3252 doesn't delete 100% of entries and some slip through it has no effect on processing pipeline updates. This also means existing instances don't need to clean up processed data to avoid #3250.
It'd be nice if we could rewrite the single big SQL update statement into chunks so we can do some informative logging of intermediate results to help with debugging updates that are in process, see #3249 - but that's for another time.
Once we have foreign key constraints we can probably remove the inner join again.
Testing
I tested deleting some sequences then updating prepro version and it worked (PR #3256)