(This description won't make sense until the Conductor PR is merged.)
In FileOnDiskMixin.delete_from_disk_and_database, the database deletion comes after the disk and archive deletion. This was changed in the Conductor PR; the reason is that the remove_downstreams option to the disk and archive deletion routines requires everything to still be in the database in order to figure out what the downstreams are.
Most of the time, this will be fine. However, if something goes wrong, it's possible we'll be left with something in the database that is not on the archive. Technically, this is an integrity error, as anything with an md5sum in the database is supposed to be on the archive.
Refactor this function to first recursively figure out everything that needs to be deleted, and then non-recursively delete from (in order) database, disk, archive.
(This description won't make sense until the Conductor PR is merged.)
In
FileOnDiskMixin.delete_from_disk_and_database, the database deletion comes after the disk and archive deletion. This was changed in the Conductor PR; the reason is that theremove_downstreamsoption to the disk and archive deletion routines requires everything to still be in the database in order to figure out what the downstreams are.Most of the time, this will be fine. However, if something goes wrong, it's possible we'll be left with something in the database that is not on the archive. Technically, this is an integrity error, as anything with an md5sum in the database is supposed to be on the archive.
Refactor this function to first recursively figure out everything that needs to be deleted, and then non-recursively delete from (in order) database, disk, archive.