When the bag updater was scanning for new bags, if it encountered an exception while scanning, that can cause the entire operation to abort and not update or add any bags.
That's not necessarily a bad thing -- we don't want the database to end up in a bad state if it encounters some broken data -- but there were two particular exceptions causing it to stop that should be handled better.
First, this MR increases the maximum length of bag names to 255 characters, which is the maximum length allowed by most Linux filesystems. It will also truncate files with longer names down to 255 characters. This will stop it from throwing an exception for bags with long names.
Second, after updating is done, it attempts to remove bag files that are missing on the filesystem from the database. If this threw an exception, it would roll back adding any new bags even if there was nothing wrong with them. This moves the removal process into its own transaction so that new bags will still be added even if there are issues removing missing ones.
When the bag updater was scanning for new bags, if it encountered an exception while scanning, that can cause the entire operation to abort and not update or add any bags.
That's not necessarily a bad thing -- we don't want the database to end up in a bad state if it encounters some broken data -- but there were two particular exceptions causing it to stop that should be handled better.
First, this MR increases the maximum length of bag names to 255 characters, which is the maximum length allowed by most Linux filesystems. It will also truncate files with longer names down to 255 characters. This will stop it from throwing an exception for bags with long names.
Second, after updating is done, it attempts to remove bag files that are missing on the filesystem from the database. If this threw an exception, it would roll back adding any new bags even if there was nothing wrong with them. This moves the removal process into its own transaction so that new bags will still be added even if there are issues removing missing ones.
Fixes #157