cardio data -- raw vs normalized counts

[donovanr]

For the cardio data Tanya says:

Counts are divided by total counts from cell and multiplied by a scaling factor (10000), then log1p of that. Raw counts are also stored in the anndata object (I think in .raw)

I think we should use the raw counts and explicitly code the normalization so that we know exactly how to back out counts to compare to FISH down the line.

[gregjohnso]

Agreed

We should solicit for good ideas on how to normalize across FISH and transcriptomics data.

[heeler]

I'm confused are you looking to translate across experiments or are you looking for normalization strategies for the input data for the model. Normalization strategies all have their own quirks and biases.

[donovanr]

• ok i'm going to push towards raw counts -> documented-in-code normalization recipe -> model inputs
• its a big topic and there's always trade-offs, for DL approaches it actually shouldn't matter as much as it does for other methods though, since the network is so flexible.
• the main priorities for me are transparency and reproducibility, with simplicity and general utility following those.
• scanpy has some built-in recipes based on recent high-profile papers
• honestly i'd be surprised if we need to get fancier than:
  • per cell normalization
  • log the counts
  • center and scale each gene to unit variance
• i'm actually not sure who to ask for advice -- who doing DL + transcriptomics that's an expert in both? not too many people. maybe we could ask on the cziscience or hca slack.