Consolidate all results

[jyaacoub]

Details

 ```python import logging from matplotlib import pyplot as plt from src.analysis.figures import prepare_df, fig_combined, custom_fig dft = prepare_df('./results/model_media/model_stats.csv') dfv = prepare_df('./results/model_media/model_stats_val.csv') models = { 'DG': ('nomsa', 'binary', 'original', 'binary'), 'esm': ('ESM', 'binary', 'original', 'binary'), # esm model 'aflow': ('nomsa', 'aflow', 'original', 'binary'), # 'gvpP': ('gvp', 'binary', 'original', 'binary'), 'gvpL': ('nomsa', 'binary', 'gvp', 'binary'), # 'aflow_ring3': ('nomsa', 'aflow_ring3', 'original', 'binary'), 'gvpL_aflow': ('nomsa', 'aflow', 'gvp', 'binary'), # 'gvpl_esm':('ESM', 'binary', 'gvp', 'binary'), # 'gvpL_aflow_rng3': ('nomsa', 'aflow_ring3', 'gvp', 'binary'), #GVPL_ESMM_davis3D_nomsaF_aflowE_48B_0.00010636872718329864LR_0.23282479481785903D_2000E_gvpLF_binaryLE # 'gvpl_esm_aflow': ('ESM', 'aflow', 'gvp', 'binary'), } fig, axes = fig_combined(dft, datasets=['davis', 'kiba', 'PDBbind'], fig_callable=custom_fig, models=models, metrics=['cindex', 'mse'], fig_scale=(10,5), add_stats=True, title_postfix=" test set performance", box=True) plt.xticks(rotation=45) # fig, axes = fig_combined(dfv, datasets=['davis'], fig_callable=custom_fig, # models=models, metrics=['cindex', 'mse'], # fig_scale=(10,5), add_stats=True, title_postfix=" validation set performance", box=True, fold_labels=True) # plt.xticks(rotation=45) ```

Final models - these are the ones we will show in the paper.

    models = {
        'DG': ('nomsa', 'binary', 'original', 'binary'),
        'esm': ('ESM', 'binary', 'original', 'binary'), # esm model
        'aflow': ('nomsa', 'aflow', 'original', 'binary'),
        # 'gvpP': ('gvp', 'binary', 'original', 'binary'),
        'gvpL': ('nomsa', 'binary', 'gvp', 'binary'),
        # 'aflow_ring3': ('nomsa', 'aflow_ring3', 'original', 'binary'),
        'gvpL_aflow': ('nomsa', 'aflow', 'gvp', 'binary'),
        # 'gvpl_esm':('ESM', 'binary', 'gvp', 'binary'),
        # 'gvpL_aflow_rng3': ('nomsa', 'aflow_ring3', 'gvp', 'binary'),
        #GVPL_ESMM_davis3D_nomsaF_aflowE_48B_0.00010636872718329864LR_0.23282479481785903D_2000E_gvpLF_binaryLE
        # 'gvpl_esm_aflow': ('ESM', 'aflow', 'gvp', 'binary'),
    }

[jyaacoub]

FIG 1

DATASET INFO

TABLE COUNTS

#### FULL TABLE COUNTS: ``` | Dataset | Protein | Compound | Total Binding Entities | |-----------|-----------|------------|-------------------------| | davis | 442 | 68 | 30056 | | kiba | 229 | 2111 | 118254 | | pdbbind | 3889 | 12639 | 19443 | ``` #### USED TABLE COUNTS: Due to memory limitations a couple records were excluded from our runs this is the full count that were actually used. ``` Dataset Protein Compound Total Binding Entities 0 davis 439 68 29852 1 kiba 226 2111 117590 2 pdbbind 3785 10950 16265 ```

SEQUENCE LENGTH DISTRIBUTION

#### non-overlayed or normalized plot  #### normalized and overlayed plot 

MODEL RESULTS

All (except pocket) results - 2x3 - MSE and Cindex


Stratified with pocket results

[jyaacoub]

FIG 3 - Platinum Dataset

DATASET INFO

TABLE COUNTS

``` Unique protein sequence counts: 860 Unique protein IDs: 361 Unique ligand counts: 197 Total records: 1962 ```

Distribution for the number of mutations per protein


pkd distributions (stratified by # of mutations)

 

Model results

Raw predictive performance

This plot shows the ability for the model to just predict the pkd given the protein sequence and ligand SMILES

Delta predictive performance

Instead of looking at absolute predictive performance this plot show how well the model is able to predict the delta between a mutated and unmutated sequence.