Implement and test GVP alternative

[jyaacoub]

Minimal code to get features for a sample protein (see commit 72c2855):

# %%
from prody import fetchPDB

fetchPDB('10gs', compressed=False)

# %%
from src.utils.residue import Chain
c = Chain('10gs.pdb', grep_atoms={'CA', 'N', 'C'})
# %%
import logging
logging.getLogger().setLevel(logging.DEBUG)

c.getCoords(get_all=True).shape # (N, 3)

# %%
from src.data_prep.feature_extraction.gvp import GVPFeatures

gvp_f = GVPFeatures()

# %%
f = gvp_f.featurize_as_graph('10gs', c.getCoords(get_all=True), c.sequence)
# %%

[jyaacoub]

Figure shows good performance with GVPL, GVPP not so much.

[jyaacoub]

Updated plot with new aflow results from https://github.com/jyaacoub/MutDTA/commit/f6cd150faeb5de578a838fe275b4def7a63e07ff

After more extensive hyperparameter tuning since it seemed weird that it had such high variance.

Code to generate figure + remove old results:

# %%
import logging
from typing import OrderedDict

import seaborn as sns
from matplotlib import pyplot as plt
from statannotations.Annotator import Annotator

from src.analysis.figures import prepare_df, custom_fig, fig_combined

df = prepare_df()
sel_dataset = 'PDBbind'
exclude = []
sel_col = 'cindex'

# removing first instance of aflow, only keeping second more tuned model
idx = df[df.run.str.contains('DGM_PDBbin.*?D_nomsaF_aflowE_128B_0.00012')].index
df.drop(idx, inplace=True)
# %%
models = {
    'DG': ('nomsa', 'binary', 'original', 'binary'),
    'aflow': ('nomsa', 'aflow', 'original', 'binary'),
    'aflow_ring3': ('nomsa', 'aflow_ring3', 'original', 'binary'),
    # 'gvpP': ('gvp', 'binary', 'original', 'binary'),
    # 'gvpL': ('nomsa', 'binary', 'gvp', 'binary'),
    'gvpL_aflow': ('nomsa', 'aflow', 'gvp', 'binary'),
    'gvpL_aflow_rng3': ('nomsa', 'aflow_ring3', 'gvp', 'binary'),
}

# %%
fig, axes = fig_combined(df, datasets=['PDBbind'], fig_callable=custom_fig,
             models=models, metrics=['pearson', 'cindex', 'mse', 'mae'],
             fig_scale=(8,5))
plt.xticks(rotation=45)

# %%

[jyaacoub]

Davis performance

• I suspect that the poor performance of GVPL_aflow on Davis is due to the low ligand count in Davis, this makes it harder for it to generalize well to unseen ligands.
• Due to the increased complexity of the GVP branch for the ligand it is more likely to overfit under such few data points.

[jyaacoub]

Results for all datasets:

• [x] Note we have to retrain all GVPL_aflow models due to logic error causing missed _edge_weightopt (see https://github.com/jyaacoub/MutDTA/commit/8c89a24318b41c66e9007c65ff55e16cb5ddb059)...

[jyaacoub]

Adding ESM to davis GVPL_aflow model doesn't really help much