Datasets with incorrect `is_primary_data=True`

[pablo-gar]

From @jahilton

@bkmartinjr surfaced observations in the Census that are is_primary_data:True & have matching count data with another observation that is_primary_data:True.

• 283d65eb-dd53-496d-adb7-7570c7caa443 - curation error
  • [x] 10,411 obs patched to is_primary_data:False
  • [x] Revision published in cellxgene?
• 939769a8-d8d2-4d01-abfc-55699893fd49 - curation error
  • [x] 100,403 obs patched to is_primary_data:False
  • [x] Revision published in cellxgene?
• bd5230f4-cd76-4d35-9ee5-89b3e7475659 & 44531dd9-1388-4416-a117-af0a99de2294 - SUPER sneaky data reuse
  • [x] 3,127 obs patched to is_primary_data:False in bd5230f4-cd76-4d35-9ee5-89b3e7475659
  • [x] Revision published in cellxgene?
• 03cdc7f4-bd08-49d0-a395-4487c0e5a168 - unexpected overlap from cell population subset Datasets
  • [x] 3,662 obs patched to is_primary_data:False + 2,891 obs patched to is_primary_data:True
  • [x] Revision published in cellxgene?
• 3a2af25b-2338-4266-aad3-aa8d07473f50 - unexpected overlap from cell population subset Datasets
  • [x] 1108 obs patched to is_primary_data:False
  • [x] Revision published in cellxgene?
• 120e86b4-1195-48c5-845b-b98054105eec (12 obs) - duplication within dataset - researcher error
  • [x] 6 obs removed from the Dataset
  • [x] Revision published in cellxgene?
• a48f5033-3438-4550-8574-cdff3263fdfd (8 obs) - duplication within dataset, requires further digging
  • [x] Revision published in cellxgene?
• 4796c91c-9d8f-4692-be43-347b1727f9d8 (4,886 obs) - duplication within dataset - researcher error
  • will need to circle back to contributor
  • [ ] Revision published in cellxgene?

[jahilton]

The final 2 datasets to investigate/correct are from the same group. We have reached out regarding the more significant one (4,886 obs) and they are investigating. Once a path forward is determined, we will surface the other (8 obs)

[brianraymor]

@jahilton - is there any update for the final 2 datasets ?

[jahilton]

we did inform the group of the second dataset. they are investigating both

[brianraymor]

@jahilton - is there any update for the final 2 datasets ?

[jahilton]

Heard from the contributor linked to a48f5033-3438-4550-8574-cdff3263fdfd (case with 8 duplicated obs)

maybe there are some errors in transferring the sample labels when we merge the datasets for each sample, which also results in duplicated cells. We are now working on mapping those messed-up cells to their correct sample labels, which would inform us where the duplicated cells are coming from

[jahilton]

Update on a48f5033-3438-4550-8574-cdff3263fdfd - they provided updated files that no longer have duplicated obs data, but there were other issues in the metadata that are being ironed out

[jahilton]

Update on a48f5033-3438-4550-8574-cdff3263fdfd - revised & Published (cc @pablo-gar )

[jahilton]

Still no word from the contributors of our last case on if they have found the solution. We reached out to them again today to check in.

[jahilton]

We have reached out again to the contributors of our last case. A deadline of Dec 12 has been set to resolve the issue or the Collection will be deleted.