Now that datasets are configured to use soft deletion, we no longer need to support cascade deletion for dataset_column which is very inefficient and caused the data ingestion pipeline to take hours to complete. If a dataset is soft deleted, we can just keep the dataset_column entities, we don't need to delete them because the original dataset is marked as deleted already.
Screenshots or Videos (if applicable)
n/a
Related Issues
Closes #341
Test Plan
Run yarn sync-schema
Open the scout db in postgres, look at the dataset_column schema and verify that the CASCADE constraint has been removed from the datasetId foreign key constraint.
Checklist Before Requesting a Review
[x] I have performed a self-review of my code
[x] My code follows the Style Guidelines and Best Practices outlined in the project wiki
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made changes to the documentation, if applicable
[x] My change generates no new warnings or failed tests
[ ] If it is a core feature, I have added thorough tests
Summary
Now that datasets are configured to use soft deletion, we no longer need to support cascade deletion for dataset_column which is very inefficient and caused the data ingestion pipeline to take hours to complete. If a dataset is soft deleted, we can just keep the dataset_column entities, we don't need to delete them because the original dataset is marked as deleted already.
Screenshots or Videos (if applicable)
n/a
Related Issues
Closes #341
Test Plan
yarn sync-schemascoutdb in postgres, look at thedataset_columnschema and verify that theCASCADEconstraint has been removed from thedatasetIdforeign key constraint.Checklist Before Requesting a Review