Ensure subsampling flags are always reset on job completion (SCP-5774)

[bistline]

BACKGROUND & CHANGES

This update addresses a corner case where some users uploaded classic SCP-formatted metadata/clustering files that were eligible to be subsampled, but upon job completion the is_subsampling flag was not correctly reset. This led to the files being "stuck" and unable to be removed by the end user, requiring direct database intervention. This was caused by a corner case where the job did not fail, but also did not generate any subsampled data. Now, the is_subsampled flag is always reset upon job completion, unless the underlying clustering object has been deleted (in which case file deletion has already happened).

MANUAL TESTING

1. Boot all services, log in, and create a new study
2. Select the "classic" upload UX
3. Upload the following two files:
   • Metadata: scp_metadata.v2.col_1_to_13.tsv&project=broad-singlecellportal-staging)
   • Clustering: umap_coordinates.tsv&project=broad-singlecellportal-staging)
4. In a separate terminal, open a Rails console session and load the study:

   study = Study.last

5. Wait until you see the success email for the cluster/metadata ingest, and then load the extracted clustering object:

   cluster = study.cluster_groups.first

6. Confirm the cluster is in the process of subsampling:

   cluster.is_subsampling
   => true

7. Wait for the success email for the subsampling run, then reload the cluster and confirm it is no longer subsampling:

   cluster.reload
   cluster.is_subsampling
   => false

8. Back in the upload UI, delete both files and confirm you receive a success dialog stating the file was deleted.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 69.70%. Comparing base (e4ea323) to head (55c0b73). Report is 26 commits behind head on development.

Additional details and impacted files

[](https://app.codecov.io/gh/broadinstitute/single_cell_portal_core/pull/2124?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=broadinstitute) ```diff @@ Coverage Diff @@ ## development #2124 +/- ## =============================================== + Coverage 69.62% 69.70% +0.07% =============================================== Files 327 328 +1 Lines 27572 27644 +72 Branches 2289 2298 +9 =============================================== + Hits 19198 19269 +71 - Misses 8240 8241 +1 Partials 134 134 ``` | [Files with missing lines](https://app.codecov.io/gh/broadinstitute/single_cell_portal_core/pull/2124?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=broadinstitute) | Coverage Δ | | |---|---|---| | [app/models/ingest\_job.rb](https://app.codecov.io/gh/broadinstitute/single_cell_portal_core/pull/2124?src=pr&el=tree&filepath=app%2Fmodels%2Fingest_job.rb&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=broadinstitute#diff-YXBwL21vZGVscy9pbmdlc3Rfam9iLnJi) | `45.36% <100.00%> (+0.78%)` | :arrow_up: | ... and [9 files with indirect coverage changes](https://app.codecov.io/gh/broadinstitute/single_cell_portal_core/pull/2124/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=broadinstitute)