search

broadinstitute / grit-benchmark

Benchmarking a metric used to evaluate a perturbation strength
BSD 3-Clause "New" or "Revised" License
5 stars 5 forks source link

Cell health: CERES vs. Grit #1

Open jt-neal opened 3 years ago

jt-neal commented 3 years ago

CRISPR_(Avana)_Public_20Q3_subsetted (1).xlsx

gwaybio commented 3 years ago

got it! If possible, can you point me to the full resource?

i.e. - not subset by cell line or 230 gene set

gwaybio commented 3 years ago

nvm, I think i found it:

URL: https://ndownloader.figshare.com/files/24613292 Source: https://depmap.org/portal/download/ License: https://creativecommons.org/licenses/by/4.0/ Changes: None

gwaybio commented 3 years ago

This figshare resource may also be helpful: https://figshare.com/articles/dataset/CERES_-_Meyers_Bryan_et_al_Nature_Genetics_2017_/5319388

gwaybio commented 3 years ago

In #2 I add the code, data, and figures to compare grit and CERES scores in the Cell Health project.

Results

cell_health_grit_ceres_comparison

cell_health_barcode_control_comparison

Interpretation

Generally, Grit and CERES scores are negatively correlated. A low CERES score indicates high gene essentiality, whereas a high Grit score indicates difference to NT control and reproducibility to the same CRISPR gene target. We do see quite a bit of off diagonal in the top figure. I think this is a cool result, suggesting that high Grit can detect signal not associated with essentiality (i.e. not necessarily a viability phenotype)

The two facets of the top figure represent the control used in the grit calculation. Specifically: 

{
  'cutting_control': ['Chr2-1', 'Chr2-4', 'Chr2-5', 'Chr2-2', 'Luc-1', 'LacZ-3', 'Luc-2', 'LacZ-2', 'Chr2-3', 'Chr2-6'],
 'perturbation_control': ['EMPTY']
}

see broadinstitute/cell-health#2 for more discussion about these control perturbations.

It is also interesting to see the majority of the points in the bottom figure fall above the dotted red line. This means that grit is generally higher using an Empty control than a cutting control. This makes sense since the perturbations cut, and therefore would form profiles that are closer to the cutting controls.

jt-neal commented 3 years ago

sorry for the naive question, but how do I access the raw data for Cell Health grit scores?

gwaybio commented 3 years ago

no worries! The scores are here: https://github.com/broadinstitute/grit-benchmark/blob/8144f3fc6f6134549ebeaeaec4b433c12909e9fd/1.calculate-metrics/results/cell_health_grit.tsv

jt-neal commented 3 years ago

in this file what does the "barcode_control" column indicate? It has both "cutting control" and "perturbation control" rows for every guide.

gwaybio commented 3 years ago

I used two different types of controls in the grit calculation - guides that cut, and no treatment. I indicate the specific guides used in the "Interpretation" section above.

jt-neal commented 3 years ago

ah - now I understand - thanks!