DataBiosphere / azul

Metadata indexer and query service used for AnVIL, HCA, LungMAP, and CGP
Apache License 2.0
7 stars 2 forks source link

Index updated projects into `lm3` on `prod` for LungMAP #4921

Closed theathorn closed 1 year ago

theathorn commented 1 year ago

There is an update to "Drop-seq of mouse lung on postnatal day 1" on row 11 of the "LungMAP Prod" tab of the TDR Datasets and snapshots spreadsheet.

Create a new lm3 catalog (with 5 total projects) so we can evaluate the update without affecting the current lm2 production release.

theathorn commented 1 year ago

From Slack thread: "A supplemental file (PND1.Loom was remove). Donors -> FASTQ files were added. An H5AD was created and linked to FASTQ files."

hannes-ucsc commented 1 year ago

Sounds like nothing that could break our indexer. Just add the catalog and request review by LM in Slack thread when done.

hannes-ucsc commented 1 year ago

Snapshot name is wrong. Should end in _lm3.

theathorn commented 1 year ago

I've asked the Broad to update the snapshot name.

theathorn commented 1 year ago

New snapshot is lungmap_prod_1bdcecde16be420888f478cd2133d11d__20220308_20230126_lm3.

theathorn commented 1 year ago

Per email from Nathan Salomonis: "We had our weekly LungMAP DCC Administrative meeting and we reviewed the new Data Browser entry the Mouse CellRef.

Perhaps I missed an email to review this but the project description and title are not appropriate for this submission.

https://data-browser.lungmap.net/explore/projects/1bdcecde-16be-4208-88f4-78cd2133d11d?catalog=lm3

Currently the title of this is:

cchmc_Xu_mouse_lung_p1_dropseq

but the name and description should reflect the paper which includes many mouse time-points. I have edited the manuscript description to exclude the human callouts:

Title: Guided construction of a single cell reference (CellRef) for mouse lung

Description: Accurate cell type identification is a key and rate-limiting step in single cell data analysis. Single cell references with comprehensive cell types, reproducible and functional validated cell identities, and common nomenclatures are much needed by the research community to optimize automated cell type annotation and facilitate data integration, sharing, and collaboration. In the present study, we developed a novel computational pipeline to utilize the LungMAP CellCards as a dictionary to consolidate single-cell transcriptomic datasets of 17 mouse lung samples and constructed “LungMAP CellRef” and “LungMAP CellRef Seed” for both normal human and mouse lungs. “CellRef Seed” has an equivalent prediction power and produces consistent cell annotation as does “CellRef” but improves computational efficiency and simplifies its utilization for fast automated cell type annotation and online visualization. This atlas set incorporates 40 mouse well-defined lung cell types catalogued from diverse developmental time points. Using independent datasets, we demonstrated the utility of our CellRefs for automated cell type annotation analysis of both normal and disease lungs. User-friendly web interfaces were developed to support easy access and maximal utilization of the LungMAP CellRefs. LungMAP CellRefs are freely available to the pulmonary research community through fast interactive web interfaces to facilitate hypothesis generation, research discovery, and identification of cell type alterations in disease conditions."

We will need CCHMC to update the project metadata and then re-ingest/snapshot/index the lm3 catalog. LungMAP prod default catalog remains at lm2 until this issue is resolved.

theathorn commented 1 year ago

Now available: 1 updated project on row 12 (lungmap_prod_1bdcecde16be420888f478cd2133d11d20220308_20230207_lm3) and 1 new project on row13 (lungmap_prod_6135382f487d4adb9cf84d6634125b6820230207_20230207_lm3) for 6 total projects.

hannes-ucsc commented 1 year ago

~For demo, show one new project and total of six projects.~

dsotirho-ucsc commented 1 year ago

lm3 catalog was not indexed on prod due to failures during IT https://gitlab.azul.data.humancellatlas.org/ucsc/azul/-/jobs/34962

achave11-ucsc commented 1 year ago

@hannes-ucsc: "LungMAP team reports that staging area was updated with donors that have developmental stag populated. Waiting for an updated snapshot."

theathorn commented 1 year ago

New snapshot in row 13 is lungmap_prod_6135382f487d4adb9cf84d6634125b68__20230207_20230228_lm3.

achave11-ucsc commented 1 year ago

PR against develop and include in next promotion.

achave11-ucsc commented 1 year ago

Assignee to monitor slack thread for updated snapshot.

theathorn commented 1 year ago

New snapshot in row 15 is lungmap_prod_6135382f487d4adb9cf84d6634125b68__20230207_20230314_lm3.

bvizzier-ucsc commented 1 year ago

Approved by Eric Bardes.