SCP1289 - Impaired local intrinsic immunity to SARS-CoV-2 infection in severe COVID-19

[idazucchi]

Project short name:

Shalek-Human-SeqWellS3

Primary Wrangler: @arschat

Secondary Wrangler: Ida

Associated files

• Google Drive: folder

Published study links

• Paper: Impaired local intrinsic immunity to SARS-CoV-2 infection in severe COVID-19

• Accessioned data: https://singlecell.broadinstitute.org/single_cell/study/SCP1289

• Ingest project

Key Events

• [x] Convert published metadata to HCA spreadsheet
• [x] Manually curate dataset to meet HCA metadata standard
• [x] Collect any matrix and cell-type annotation files
• [x] Are the analysis files suitable for CellxGene? If something is missing get in touch with the authors to request it
• [x] Upload sheet to validate metadata
• [x] Transfer raw files to ingest to validate data files
• [x] Check linking using ingest graph validator
• [x] Ask the Secondary Wrangler for an end-to-end review of the project. Ask the Expertise Wrangler to review specific tabs if needed
• [x] Submit dataset to Production
• [ ] Complete the Export SOP
• [ ] Convert project data to SCEA format following the SCEA conversion SOP if appropriate

[arschat]

Will use 1 cell suspension for each specimen after this slack discussion.

[arschat]

After submitting, get the following error in ingest: Project is Invalid. Please go back and edit the project. * should NOT have additional properties at root of document

Reason: bionetwork fields in spreadsheet was misspelled as project.hca_bionetwork instead of project.hca_bionetworks

[anu-shiva]

edit the project fields through API and to edit the template

[arschat]

Deleted wrong project fields through API:

proj_url = 'https://api.ingest.archive.data.humancellatlas.org/projects/64267858c99baf0c6c9f23bd'
result = requests.get(proj_url).json()
del result['content']['hca_bionetwork']
response = requests.post(proj_url, headers=headers_json, json=result)

Should not use post, use patch command instead

And resubmitted with correct fields.

Still got error since "Lung Atlas" is not in the hca_tissue_atlas enum, but decided to skip schema patch update and do not fill the hca_tissue_atlas and hca_tissue_atlas_version since it has not been decided yet how the atlas is going to be named.

[arschat]

New submission is now graph valid, and ready for secondary review.

[idazucchi]

Hi Arsenios! The dataset looks great I only have two comments:

Specimen

• I'd suggest cryopreservation, other since the cells are not dead

Analysis protocol

• you can add the derivation process

Analysis file

• there is no good ontology label for UMAP in HCAO right now, maybe it would be worth it to open an ontology ticket and request a new term
• I think you swapped the protocols for 20210220_NasalSwab_NormCounts.txt and 20210220_NasalSwab_RawCounts.txt

[arschat]

CellxGene wrangling requirements

• [X] Raw count matrix: 20210220_NasalSwab_RawCounts.txt
• [X] Normalised count matrix: 20210220_NasalSwab_NormCounts.txt
• [X] cell type annotations (cell type linked to barcode): 20210220_NasalSwab_UMAP.txt
• [ ] ENSEMBL ID of genes: MISSING
• [X] x-y coordinates (tsne, umap, pca): 20210220_NasalSwab_UMAP.txt

(genes are in gene symbol format in the count matrices)

[arschat]

Applied sec review suggestions (thanks Ida), and submitted. Input form to be filled when exported.

[arschat]

Input form sent

[ESapenaVentura]

Updated with hca tissue atlas and version

[ESapenaVentura]

Error in R28 - Neither project nor supplementary file metadata have submission_date stablished (Value: null). This may have been a side effect of programmatic updates.

Actions:

• [x] Update staging area with submission date (Take from submission entity) for both project and supplementary file metadata
• [ ] Update ingest with that same information for project

[amnonkhen]

I believe we have a gap in our validation process. Our metadata validation should have caught this error. Please create a bug ticket.

[ESapenaVentura]

yep - system fields are not validated because they are generated on the go

[ESapenaVentura]

Operational fix - Overwriting metadata in the staging area

# Project metadata
gsutil cat gs://broad-dsp-monster-hca-prod-ebi-storage/prod/111d272b-c25a-49ac-9b25-e062b70d66e0/metadata/project/111d272b-c25a-49ac-9b25-e062b70d66e0_2023-05-22T10:31:35.590000Z.json | jq -jc '.provenance.submission_date = "2023-03-31T06:06:350Z"' > 111d272b-c25a-49ac-9b25-e062b70d66e0_2023-05-22T10:31:35.590000Z.json
gsutil cp 111d272b-c25a-49ac-9b25-e062b70d66e0_2023-05-22T10:31:35.590000Z.json gs://broad-dsp-monster-hca-prod-ebi-storage/prod/111d272b-c25a-49ac-9b25-e062b70d66e0/metadata/project/111d272b-c25a-49ac-9b25-e062b70d66e0_2023-05-22T10:31:35.590000Z.json

# Supplementary file (Spreadsheet)

gsutil cat gs://broad-dsp-monster-hca-prod-ebi-storage/prod/111d272b-c25a-49ac-9b25-e062b70d66e0/metadata/supplementary_file/9ef0a3c2-9d3b-52d0-8ae8-3f968da4c390_2023-05-22T11:15:30.018000Z.json | jq -jc '.provenance.submission_date = "2023-03-31T06:06:350Z"' > 9ef0a3c2-9d3b-52d0-8ae8-3f968da4c390_2023-05-22T11:15:30.018000Z.json
gsutil cp 9ef0a3c2-9d3b-52d0-8ae8-3f968da4c390_2023-05-22T11:15:30.018000Z.json gs://broad-dsp-monster-hca-prod-ebi-storage/prod/111d272b-c25a-49ac-9b25-e062b70d66e0/metadata/supplementary_file/9ef0a3c2-9d3b-52d0-8ae8-3f968da4c390_2023-05-22T11:15:30.018000Z.json

[ESapenaVentura]

This fixes the issue for the strict timeline

[ofanobilbao]

@ESapenaVentura to confirm if we need to do the changes in Ingest. Otherwise this ticket could move to close right?

[ofanobilbao]

Was this resolved by Amnon as part of the R28 errors remaining to be sorted in Ingest that Amnon did last week? @ESapenaVentura ? Trying to understand if we can close this

[idazucchi]

missing submissionDate was add back to ingest by Amnon - this ticket can be closed!