search

broadinstitute / 2020-06-01-Evidence-of-state-switching-in-single-cell-drug-response-Broad

0 stars 2 forks source link

Harmony batch effect correction #3

Open gwaybio opened 3 years ago

gwaybio commented 3 years ago

In #2 @hkhawar applied whitening to try to correct for batch effects. This issue will track the experiment/analysis of applying Harmony to the same DMSO profiles in #2

@sMyn42 will apply Harmony to the single cell profiles in two distinct ways:

  1. Traditional Harmony
  2. Inverse transform Harmony

In both ways, we will use Harmony to adjust for the well and plate level variation.

Visualization

We will apply umap to both traditional and inverse Harmony and visualize the resulting coordinates.

Batch effect detection

We will also apply kBet to the following data transformations.

  1. Normalized single cell profiles
  2. Whitened single cell profiles
  3. Harmonized single cell profiles
  4. Inverse harmonized single cell profiles

Analysis Output

By the end of the analysis, we will have the following (in addition to the results in #2 ):

  1. Two UMAP figures (traditional and inverse Harmony)
  2. Two forms of harmonized single cell morphology profiles (traditional and inverse Harmony)
  3. kBET "rejection rate" figures for all data transformations
  4. Decision about which single cell profiles to use for downstream analyses
gwaybio commented 3 years ago

@hkhawar - can you add an experiment label to this issue?

hkhawar commented 3 years ago

@gwaygenomics Its done

gwaybio commented 3 years ago

@sMyn42

Repository Location on EC2

let's clone the fork of your repo here: /home/ubuntu/efs/2015_10_05_DrugRepurposing_AravindSubramanian_GolubLab_Broad/workspace/software

Location of single cell files

Unnormalized

@hkhawar documents the location of the _dmso.csv files for the five plates in #1

Whitened

@sMyn42 - it looks like the whitened files are never written to disk (see this notebook). We will need them written to disk since we will be analyzing them downstream in various ways.

sMyn42 commented 3 years ago

@gway @hkhawar Here's my plan for getting this done:

Broad1

In addition, when writing the whitened files to disk as part of the plan, would it be best to edit the notebook that deals with whitening (in order to write the files to disk) in place or to make a copy and edit that?

hkhawar commented 3 years ago

@sMyn42 My suggestion is to edit existing notebook for your whitening strategy

gwaybio commented 3 years ago

Looks like a solid strategy @sMyn42 - so that I can confirm my understanding, the dotted lines in the above graph indicates that these UMAP figures exist already (the ones @hkhawar already implemented), correct?

sMyn42 commented 3 years ago

Yes

On Thu, Sep 24, 2020 at 12:34 PM Greg Way notifications@github.com wrote:

Looks like a solid strategy @sMyn42 https://github.com/sMyn42 - so that I can confirm my understanding, the dotted lines in the above graph indicates that these exist already (the ones @hkhawar https://github.com/hkhawar already implemented), correct?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/broadinstitute/2020-06-01-Evidence-of-state-switching-in-single-cell-drug-response-Broad/issues/3#issuecomment-698454782, or unsubscribe https://github.com/notifications/unsubscribe-auth/AIPDAP2GYZXCA43TMRIWDADSHNYIXANCNFSM4RL5P2BA .

hkhawar commented 3 years ago

Harmony_transformed features_plate

Harmony_transformed features_well

Harmony_inverse_transformed features_plate

Harmony_inverse_transformed features_well

ZCA_whitened_plate

ZCA_whitened_well

No_whitening_plate

No_whitening_well

hkhawar commented 3 years ago

@gwaygenomics @shntnu Have a look at the plots of harmony batch corrected data. Though all the plates are not aligned perfectly but if we plot well wise we do see there is no segregation of well profiles in Umap space as we do observe in ZCA whitened data

sMyn42 commented 3 years ago

This is really cool! I guess inverse harmony just got wiped out by the decomposition of the UMAP, but that ZCA transformation picture is r/dataisbeautiful worthy.

shntnu commented 3 years ago

The Harmony batch corrected version certainly looks promising. But I stick to my recommendation in https://github.com/broadinstitute/2020-06-01-Evidence-of-state-switching-in-single-cell-drug-response-Broad/issues/2#issuecomment-721870683 i.e. consider just one plate and do all the analysis within the plate (because it appears that the well-position effect may not be as strong as we thought).