Replace hard-delete with soft-delete when a dataset can no longer be found in Socrata. This marks a dataset as deleted without actually removing it. This allows us to still show this dataset in collections and allows us to restore the dataset if the dataset becomes available in Socrata. This happens frequently because Socrata's API regularly drops dataset ids during batch calls, so it's not 100% reliable to always retrieve every dataset that exists, so it's common for a dataset to appear deleted during one data ingestion process, only to then be restored in the next one. If we used hard-deletes this would be an incredibly inconvenient process.
Screenshots or Videos (if applicable)
n/a
Related Issues
Closes #342
Test Plan
Run yarn seed-database-dev and verify if any datasets that get deleted now have their deletedAt column populated
Checklist Before Requesting a Review
[x] I have performed a self-review of my code
[x] My code follows the Style Guidelines and Best Practices outlined in the project wiki
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made changes to the documentation, if applicable
[x] My change generates no new warnings or failed tests
[ ] If it is a core feature, I have added thorough tests
Summary
Replace hard-delete with soft-delete when a dataset can no longer be found in Socrata. This marks a dataset as deleted without actually removing it. This allows us to still show this dataset in collections and allows us to restore the dataset if the dataset becomes available in Socrata. This happens frequently because Socrata's API regularly drops dataset ids during batch calls, so it's not 100% reliable to always retrieve every dataset that exists, so it's common for a dataset to appear deleted during one data ingestion process, only to then be restored in the next one. If we used hard-deletes this would be an incredibly inconvenient process.
Screenshots or Videos (if applicable)
n/a
Related Issues
Closes #342
Test Plan
yarn seed-database-devand verify if any datasets that get deleted now have theirdeletedAtcolumn populatedChecklist Before Requesting a Review